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Highlights 


Alberta’s  Primary  Health  Care  project  was  a highly  successful  project. 
Many  people  across  regional  health  authorities  have  had  an  opportunity  to 
develop,  implement,  and  evaluate  innovative  approaches  to  improve  the 
current  health  system.  Learning 
has  revealed  new  ways  to 
improve  access  to  existing 
services.  New  models  have  been 
piloted  to  improve  quality  of  care. 

A great  deal  of  evidence  has 
been  gathered  to  support  how 
health  professionals  can  work  together  more  effectively  to  provide 
increased  continuity  of  care  and  reduced  duplication  of  service.  Project 
activity  points  toward  the  potential  for  increased  efficiency  and  reduced 
cost — significant  impacts  on  Alberta’s  health  system. 

The  Alberta  Primary  Health  Care  project  has  demonstrated  that 

> Continuity  in  care  can  be  achieved  by  creating  an  integrated  system  of 
service  delivery. 

> Integration  is  supported  through  collaborative  planning  processes  that 
include  health  professionals,  inter-agency  and  cross-sectoral 
partnerships,  and  consultation  with  stakeholder  groups. 

> Physicians  can  play  a key  leadership  role  in  catalyzing  the  participation 
of  other  health  providers. 

> Trust  and  relationships  are  central  to  team  building  among  various 
provider  groups. 


“I  would  have  lost  my  son  that  day,  if  it 
hadn't  been  for  Healthy  Families.  I don't 
know  of  anything  else  that  is  remotely  like 
it  anywhere  else. " (Healthy  Families 
Parent) 


The  Multidisciplinary  Teams  project  in 
Calgary  has  demonstrated  that  when 
physicians  work  with  nutritionists  and 
other  providers  to  transform  best 
evidence  into  best  practice,  clinical 
practice  guidelines  must  be  translated 
into  specific  and  practical  tasks  that  will  be  carried  out  at  the  front  line  of 
care.  The  project  also  demonstrated  how  internet-based  decision  support 
tools  can  help  providers  learn  about  each  others’  values  and  practice.  This 
knowledge  is  important  for  integrated  service  provision.  The  involvement 
of  front-line  practitioners  is  key. 

The  Shared  Care  project  in  Calgary  demonstrated  improved  capacity  of 
family  practitioners  to  diagnose  and  treat  mental  health  problems  of 
patients  through  collaborative  consultation  and  support  from  psychiatrists. 


Health  professionals  can 
work  together  to  improve 
continuity  of  care 
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Similarly,  the  C.O.P.E.  school-based  mental  health  project  demonstrated 
that  when  medical  professionals  work  with  teachers  and  parents, 
children’s  mental  health  problems  are  diagnosed  earlier. 

The  Lakeland  Palliative  Care  project  very  clearly  demonstrated  that  when 
a Palliative  Care  Nurse  Consultant  and  Palliative  Care  Physician 
Consultant  are  able  to  share  their  expertise  with  other  care  providers,  the 
level  of  quality  care  provided  to  palliative  patients  and  their  families  rises 
significantly.  What  is  more,  the  region  has  increased  capacity  to  support 
palliative  patients  without  them  having  to  travel  outside  the  region. 

The  Collectives  project  in  Edmonton  demonstrated  that  when  community 
pharmacists  work  with  physicians,  home  care  nurses,  and  the  patient, 
medication  compliance  among  seniors  is  improved.  This  has  important 
implications  for  our  growing  senior’s  population. 

Also,  several  projects  addressed  community  health  centre  models — some 
in  urban  environments  like  CUPS,  Alexandra,  and  the  8th  and  8th  Health 
Centres  in  Calgary  and  the  Northeast  Community  Health  Centre  in 
Edmonton,  as  well  as  others  in  outlying  areas  like  the  Airdrie  Regional 
Health  Centre.  Much  has  been  learned  about  how  community  health 
centres  must  function  to  support  a primary  health  care  focus,  that  is, 
holistic  care  to  address  the  physical,  mental,  and  spiritual  health  needs  of 
the  service  recipient.  Structures  and  processes  that  support  team  building 
and  community  involvement  are  critical. 

Project  learning  has  indicated  that  two  key  barriers  to  supporting  an 
integrated  health  system  are  information  sharing  among  providers  and 
appropriate  methods  of  payments  for  physicians  and  other  providers. 


> There  is  need  for  clinically-integrated  databases  where  on-line  real 
time  information  is  simultaneously  available  to  all  care  providers. 

> Standardized  communication  mechanisms  such  as  templates  and 
electronic  health  records  are  required  to  support  information  sharing. 

> Information  sharing  supports  collaborative  planning  and 
multidisciplinary  team  work. 


Doing  business  differently 
encourages  collaboration 
and  positively  impacts 
patients 


The  C.O.P.E.  project  in  Calgary 
demonstrated  very  clearly  that  the 
current  fee-for-service  schedule 
discourages  collaboration  among 
provider  groups.  For  example, 
psychiatrists  can  bill  for  consultation 
time  without  the  patient  being  present 
while  pediatricians  and  family  physicians  cannot.  Further,  current  fee-for- 
service  models  do  not  recognize  the  non-billable  time  a physician  may 
spend  with  clients  in  a primary  health  care  setting  (e.g.,  providing 
telephone  consultation  to  patients).  Appropriate  reimbursement  systems 
that  encourage  interdisciplinary  and  multidisciplinary  primary  health  care 
activity  need  to  be  developed  and  implemented. 
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The  CUPS  project  has  demonstrated  that  introducing  a Nurse  Practitioner 
into  an  inner  city  Health  Centre,  reduces  the  turn-away  rate  by  as  much  as 
40%.  The  Nurse  Practitioner  is  able  to  see  patients  who  do  not  require  the 
special  skills  of  a physician.  Together,  this  project  and  the  Elnora  project 
(which  introduced  a Nurse  Practitioner  into  a rural  Health  Centre)  support 
the  use  of  Nurse  Practitioners  in  primary  health  care  settings  beyond 
northern  remote  environments. 


Several  projects  emphasized  the  importance  and  benefit  of  addressing  the 
determinants  of  health  and  focusing  on  health  promotion  and  prevention 
as  key  elements  embedded  within  a primary  health  care  approach. 

Two  Healthy  Families  projects  (one  in  the  Capital  Health  Authority  and  one 
in  WestView  Regional  Health  Authority)  illustrate  the  value  and  positive 
influence  of  Family  Visitors  on  at-risk  mothers  and  their  children. 

> Parent  satisfaction  with  the  Program  was  high. 

> Fewer  pre-term  babies  and  low  birth  weight  babies  were  reported. 

> Breast  feeding  rates  improved. 

> Immunizations  were  kept  current  for  approximately  90%  of  babies  and 
80-97%  of  children  tested  were  in  the  normal  range  of  development. 


In  the  Capital  Health  region  a decrease  in  the  number  of  Healthy  Families 
infants  visiting  Emergency  Department  attests  to  the  increased  capacity  of 
parents  to  care  for  the  primary  needs  of  their  children.  What  is  more,  the 
Healthy  Families  projects  illustrate  that  ready-made  programs  need  to  be 
tailored  to  the  Alberta  context  and  that  care  providers  can  be  trained  right 
here  in  our  own  province. 


The  introduction  of  a Settlement  Nurse  on  Metis  Settlements  in  Lakeland 
Regional  Health  Authority  encouraged  community  members  to  be  more 
aware  of  how  they  could  improve  their  health.  The  Nurse  helped 
community  members  understand  what  health  services  were  available  to 
them  and  how  they  could  begin  working  and  planning  with  the  region  to 
improve  their  use  of  existing  services.  The  project  demonstrated  the 
importance  of  the  Settlement  Nurse  as  a key  link  between  the  Region  and 
Metis  Settlements.  Project  participants  said  that  the  Primary  Health  Care 
project  allowed  this  first  bridge  to  be  built. 


The  Immunization  project  in  Keeweetinok  Lakes  Regional  Health 
Authority,  a health  authority  covering  a large  geographic  area  that  has 

several  remote  communities  and  yet  a 
small  population,  developed  a successful 
strategy  to  increase  the  immunization  rates 
of  its  children.  The  project  learned  that 
making  personal  contact  with  guardians  is 
a significant,  if  not  the  most  important 
factor,  in  encouraging  adherence  to 
immunization  schedules.  Immunization 


Providing  services  to 
the  hard  to  reach  can  be 
done  efficiently  and 
effectively 
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rates  were  increased  by  more  than  20%.  In  a province  where  the  structure 
of  the  public  health  system  has  changed  substantially  since  regionalization 
in  1994,  this  finding  has  relevance  across  the  province  and  across 
Canada. 

It  is  difficult  to  discuss  primary  health  care  without  making  special  mention 
of  projects  that  contributed  to  advancing  primary  health  care  in  rural 
Alberta.  The  Healthy  Okotoks  Coalition  was  a demonstration  of  a very 
successful  community  development  strategy  that  mobilized  a group  of 
community  members  to  organize  a variety  of  healthy  primary  health  care 
options  for  their  entire  community  including 


> The  establishment  of  a 
community  garden. 

> Awareness  raising  of  fetal 
alcohol  syndrome. 

> Implementation  of  a youth 
curfew. 


> Approval  of  a skateboard 
park. 

> The  addition  of  youth 
positions  to  the  Town 
Council. 


The  Health  Authority  5 project  proved  that  travel  out  of  the  Region  to 
receive  health  services  could  be  reduced  by  introducing  mobile  teams  of 
health  providers  to  four  rural  communities.  Survey  results  indicated 
support  for  the  idea  that  multidisciplinary,  rural  health  services  and 
community  development  activities  can  have  a positive  impact  on  health 
attitudes  and  behaviours.  For  example: 

> More  people  in  the  treatment  communities  reported  improved  health- 
related  knowledge  and  complying  with  medication  regimens. 

> All  respondents  to  the  community  survey  reported  a significant 
increase  in  using  the  Internet  to  access  health  information. 

A third  rural  region  (East  Central)  has  begun  to  develop  a region-wide 
primary  health  service  delivery  system  through  the  collaborative 
involvement  of  four  rural  communities.  The  project  has  taken  the  first, 
perhaps  most  difficult,  steps  in  communicating  and  operationalizing  a 
vision  for  an  integrated  primary  health  service  system  for  the  Region. 

And  finally,  project  participants  have  demonstrated  enormous  commitment 
to  advancing  primary  health  care  in  this  province.  Physicians  have  served 
as  leaders  in  this  movement — often  volunteering  their  time.  A host  of  other 
providers  have  also  committed  time  and  energy  to  understanding  how  the 
health  system  could  be  improved.  A variety  of  tools  and  models  have  been 

developed  and  tested — to  gather 
client  information,  map  health 
services,  and  assess  the  quality 
of  family  practice  medical  clinics 
(for  example  the  Sunridge 
project).  Models  for  integrated 
palliative  care  service  delivery 
have  also  been  developed  along 


The  Alberta  Primary  Health 
Care  project  generated  a 
wealth  of  experience  and  a 
generous  legacy  of  ideas  and 
tools. 
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with  communication  templates  and  palliative  sedation  guidelines.  A 
Continuous  Quality  Improvement  Process  has  been  developed  for 
palliative  care  (Calgary  Palliative  Care  project),  as  well  as  a model  for 
school-based  mental  health  outreach  (C.O.P.E.)  and  a Shared  Care  Model 
for  Mental  Health  Care.  A Lifestyle  Risk  Reduction  program  has  been 
developed  and  tested  for  cholesterol  reduction,  and  a model  for 
Collectives  activity  between  physician,  community  pharmacist,  home  care 
nurse,  and  patient  are  ready  to  applied  in  other  environments. 

Understanding  has  been  gained  about  why  patients  use  the  Emergency 
Department  for  primary  care  reasons.  Findings  suggest  that  worry  and  a 
sense  of  urgency  drive  people  to  emergency  departments  rather  than 
convenience  and  proximity  to  home.  This  is  important  information, 
specifically  for  understanding  how  perceptions  of  urgent  health  needs 
differ  for  users,  particularly  new  Canadians.  These  many  and  varied 
examples  attest  to  the  range  of  contributions  of  project  participants. 


Still,  there  is  much  work  left  to  do.  Projects  have  identified  that  structures 
and  processes  need  to  be  developed  to  support  multidisciplinary  activities. 
Strategies  have  been  We  have  a reaj  //Ve  examp/e  Qf  improving  care  in 

developed  to  assist  the  community  in  a team  environment  The  fact 

projects  in  assessing  that  the  participants  are  trying  to  think  of  ways  to 

their  cost-effectiveness  continue  on  an  ongoing  basis  and  are  willing  to 

and  sustainability.  i“mp°™r  mT  hurdles  obviously  they want  it!" 

. (Health  Care  Provider,  Collectives  Project) 

These  are  key 

accountability  tasks  that  need  to  be  undertaken.  Project  participants  would 
say  that  they  have  just  gotten  started,  that  time  was  too  short  to  continue 
the  good  work  they  had  started.  What  is  clear  is  that  significant 
contributions  have  already  been  made  within  the  two-year  window  of  the 
Health  Transition  Fund. 


fA 
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Introduction 


The  National  Forum  on  Health  (1997)  emphasized  the  importance  of 
developing  an  integrated,  responsive,  and  effective  health  system  through 
reform  of  primary  health  care.  As  one  response  to  recommendations 
arising  from  the  Forum,  the  federal  government  created  the  $150  million 
Health  Transition  Fund  (HTF)1  which  had  a three-year  mandate  (1997- 
2000)  to  study  and  evaluate  innovative  ways  of  delivering  health  services 
and  improving  the  health  system.  Through  the  outcomes  of  national  level 
initiatives  and  provincial  and  territorial  evaluation  and  research  pilot 
projects,  the  HTF  was  designed  to  provide  evidence  about  the  feasibility  of 
implementing  various  approaches  to  reform  and  improve  Canada’s  health 
system. 

The  federal,  provincial,  and  territorial  governments  identified  four  priority 
areas  for  the  Fund.  These  included  homecare,  pharmacare,  primary  health 
care  reform,  and  integrated  delivery  of  services. 

Alberta’s  project,  which  focused  on  primary  health  care,  was  officially 
announced  in  April  1998  by  Alberta  Health  and  Wellness  Minister  Jonson 
(at  that  time),  and  by  federal  Health  Minister  Rock.  An  $1 1 million  fund 
was  dedicated  to  encourage  advancements  in  primary  health  care  by 
facilitating  the  development  and  study  of  innovative  models,  approaches, 
and  delivery  systems.  Efforts  were  directed  to  inform  the  practice  of 
primary  health  care  by  regional  health  authorities  and  others  and  also 
future  development  of  primary  health  care  planning  and  policy. 

Alberta’s  focus  on  primary  health  care  was  the  result  of  priority  setting 
processes  held  in  Alberta  prior  to  1998.  A number  of  regional  health 
authorities,  communities,  practices,  and  professional  organizations,  as 
well  as  consultation  with  key  stakeholders,  indicated  support  for  the 
enhancement  of  a primary  health  care  approach  for  both  rural  and  urban 
areas  of  the  province.  Primary  health  care  was  also  identified  as  a key 
strategy  in  Alberta  Health  and  Wellness’  Business  Plan  (1997-1998  to 
1999-2000). 

Commitment  to  this  single  priority  (i.e.,  primary  health  care)  indicated 
significant  support  on  behalf  of  Alberta  Health  and  Wellness  to  provide 
resources  and  coordination  for  the  advancement  of  primary  health  care. 
Emphasis  was  placed  on  determinants  of  health  and  population  health 
strategies,  illness/injury  prevention,  health  promotion,  and  wellness.  The 
active  involvement  of  communities  and  individuals  in  partnership  with 
providers  was  emphasized  in  decision-making  processes.  This  focus 


Support  for  Alberta’s  project  is  part  of  broader  support  provided  by  the  HTF  across  Canada  $120 
million  for  projects  at  provincial  and  territorial  levels  as  well  as  $30  million  for  projects  at  the  national 
level. 
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aligned  well  with  a view  of  primary  health  care  as  the  first  level  of  contact 
of  individuals  with  the  health  care  system — that  is,  where  health  services 
are  mobilized  and  coordinated  to  promote  health,  prevent  illness,  care  for 
common  illness,  and  manage  ongoing  health  problems  (National  Forum 
on  Health,  1997). 

Following  HTF  approval,  the  Alberta  Primary  Health  Care  project  was 
initiated  by  a provincial  Call  for  Proposals  in  April  1998.  An  external 
Selection  Advisory  Committee  reviewed  proposals  and  recommended  to 
the  provincial  Minister  of  Health  that  24  be  funded.  An  additional  3 
proposals  were  later  approved.  These  27  projects  (under  the  umbrella 
Alberta  Primary  Health  Care  Project)  were  designated  demonstration  (15 
projects),  research  and/or  evaluation  (12  projects). 

A few  months  later  (October  1998),  a key  stakeholder  consultation 
process  was  conducted  in  the  province  which  resulted  in  a Draft 
Discussion  Paper  on  Primary  Health  Care.  This  Discussion  Paper 
(D’Andrea,  1998)  presents  the  views  of  multiple  stakeholders  on  key 
components  that  should  be  considered  in  the  development  of  future  policy 
on  primary  health  care. 

Evaluation 


A key  goal  of  the  Health  Transition  Fund  was  evaluation  of  projects  to 
identify  key  learning  in  priority  areas.  All  HTF  projects  were  expected  to 
have  measurable  outcomes  and  undergo  rigorous  investigation  of  project- 
specific  evaluation  questions  and  six  areas  of  inquiry  identified  at  the 
national  level  (access,  quality,  impact  on  populations,  integration,  cost- 
effectiveness,  and  transferability1 2).  Learning  from  these  two  levels  of 
evaluation  is  expected  to  be  instructive  to  regional  health  authorities, 
professional  health  organizations,  collaborating  agencies,  and  other 
primary  health  care  stakeholders  at  the  local  level. 

Approach  to  Evaluation 

A comprehensive  evaluation  of  each  of  the  27  projects  was  undertaken. 
To  coordinate  evaluation  activity,  an  independent  consulting  firm,  Howard 
Research,  was  contracted  to  manage  the  evaluation  process.  A back-up 
consultant  (Wynne  Resources)  assisted  Howard  Research  as  part  of  the 
Evaluation  Management  Team  (EMT).  Management  duties  of  the  EMT 
included 

1 . meeting  with  individual  projects  to  clarify  project  and  evaluation  goals 

and  objectives, 


A list  of  the  six  national  dimensions  and  their  sub-questions  is  presented  in  Appendix  B. 
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2.  selecting  and  contracting  independent  evaluators  for  each  project  in 
consultation  with  Project  Directors/Managers  and  Alberta  Health  and 
Wellness, 

3.  providing  support  to  independent  evaluators, 

4.  monitoring  evaluation  progress, 

5.  reporting  progress  to  Alberta  Health  and  Wellness, 

6.  facilitating  meetings  with  all  independent  evaluators, 

7.  reviewing  independent  evaluators’  interim  and  final  evaluation  reports, 
and 

8.  analyzing  and  synthesizing  evaluations  across  the  27  projects. 

A Referent  Group  was  established  to  provide  advice  and  guidance  to  the 
EMT.  This  five-member  panel  reviewed  the  progress  of  projects  and 
discussed  critical  issues  with  the  EMT. 

Individual  projects  were  each  assigned  an  independent  evaluator  to 
conduct  an  objective  evaluation  of  project  activity  and  outcomes.  Fourteen 
teams  of  evaluators  (a  total  of  35  evaluators)  evaluated  the  27  projects. 
Since  projects  were  of  different  types  (i.e.,  demonstration,  research,  or 
evaluation),  the  role  of  independents  varied  across  projects.  However,  all 
independent  evaluators  were  encouraged  to  critically  reflect  on 
implementation  activity  and  progress  with  project  participants  so  that  valid 
and  reliable  results  would  be  achieved.  Independent  evaluators  met 
periodically  with  the  EMT  to  discuss  evaluation  issues  including  how 
project  progress  was  informing  policy  development  related  to  primary 
health  care. 

The  following  model  illustrates  the  relation  of  Alberta’s  project  to  the 
Health  Transition  Fund: 
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Methodology 


Meta-evaluation 

A modified  meta-evaluation  approach  was  used  to  conduct  the  overall 
evaluation  of  the  27  projects.  A meta-evaluation  is  often  referred  to  as  an 
evaluation  of  an  evaluation  and  serves  two  major  roles:  formative  and 
summative.  The  formative  role  is  “to  guide  the  planning  and 
implementation  of  a program  evaluation(s)”  and  the  summative  role  is  to 
assess  the  worth  or  merit  of  completed  program  evaluation(s).”3 

In  terms  of  the  formative  role,  the  EMT  was  engaged  with  the  independent 
evaluators  throughout  the  evaluation  process.  Site  visits  were  made  by  the 
EMT  with  a representative  of  Alberta  Health  and  Wellness  at  project 
initiation  and  at  various  points  throughout  the  implementation  phase.  The 
summative  task  involved  analyzing  results  in  terms  of  contribution  to  the 
six  national  dimensions  (i.e.,  access,  quality,  integration,  impact  on 
populations,  cost-effectiveness,  transferability)  and  to  advancing  primary 
health  care  in  Alberta.  The  summative  task  focused  less  on  judging  the 
worth  or  merit  of  the  independent  evaluations  than  on  deriving  as  much 
learning  as  possible  to  inform  policy  decisions  at  the  provincial  level. 

Data  Collection 


At  the  project  level,  a variety  of  qualitative  and  quantitative  methods  were 
used  to  collect  and  analyze  data  for  evaluation  purposes.  Quantitative 
methods  included  the  use  of  descriptive  and  inferential  statistics. 
Qualitative  methods  included  interviews,  surveys,  document  review, 
observation,  reviews  of  primary  and  secondary  data,  reflective  sessions, 
modeling,  chart  reviews,  literature  reviews,  and  reviews  of  clinical  records. 
Variation  in  these  methods  required  that  complimentary  methods  were 
used  to  assess  project  findings. 

Independents  were  provided  with  a guide  to  focus  their  final  analyses  of 
project  activities  and  results.  These  guidelines  included  a requirement  to 
explicate  how  project  findings  contributed  to  any  or  all  of  the  national 
dimensions.  One  important  element  of  the  guide  was  the  creation  of  a 
“Context”  section  in  the  final  independent  evaluation  reports  to  help 
readers  understand  the  environment  in  which  the  project  was  implemented 
and  the  extent  to  which  projects  could  be  replicated  elsewhere.  In  addition, 
independent  evaluators  were  asked  to  identify  implications  for  policy  that 
would  assist  in  determining  to  what  extent  project  results  informed  policy 
action  at  the  provincial  level.  These  insights  assisted  the  EMT  in 
identifying  challenges  and  barriers  to  support  and  sustain  project 
achievements. 


3 The  Joint  Committee  on  Standards  for  Educational  Evaluation.  (1994).  The  program  evaluation 
standards:  howto  assess  evaluations  of  educational  programs  (2nd  ed)  pp.  185-190. 


September  2000 


10 


Advancing  Primary  Health  Care 


Other  documentary  data  were  collected  and  examined  by  the  EMT.  These 
documents  included  the  primary  health  care  final  project  reports, 
independent  evaluation  progress  and  interim  reports,  EMT  members’ 
notes  taken  during  site  visits,  reflective/discussions,  workshops  and 
meetings  (i.e.,  with  representatives  from  Alberta  Health  and  Wellness, 
Project  Directors/Managers,  Referent  Group,  other  primary  health  care 
stakeholders),  correspondence,  faxes  and  e-mails,  and  federal/provincial 
government  documents  relating  to  primary  health  care. 

Data  Analysis 

Howard  Research  has  prepared  a set  of  background  papers  (a 
monograph  entitled  “Primary  Health  Care:  Six  Dimensions  of  Inquiry”)  that 
explore  the  theoretical  concepts  and  historical  developments  for  each  of 
the  six  dimensions  of  inquiry.4  This  monograph  has  served  a useful 
function  in  the  analysis  process  in  that  has  prepared  a foundation  on 
which  to  reflect  findings  derived  from  the  evaluation  process. 

Each  of  the  27  independent  evaluation  reports  has  been  content 
analyzed — particularly  in  relation  to  contribution  to  the  six  national 
dimensions.  A synopsis  of  each  independent  report  can  be  found  in 
Appendix  A.  These  synopses  are  limited  to  key  findings  only  and  are  not 
meant  to  replace  the  comprehensiveness  or  richness  of  the  independent 
reports. 

Three  levels  of  thematic  analyses  were  then  conducted.  Level  one 
identified  themes  within  each  project.  Second  level  analysis  involved  the 
identification  of  themes  common  within  each  of  the  six  dimensions.  And 
third  level  analysis  involved  identification  of  themes  common  across  all 
projects  including  the  six  areas  of  inquiry.  This  final  level  of  analysis 
comprises  the  Implications  section  of  this  report. 

Trustworthiness  of  Findings 

In  evaluation  issues  of  validity  and  reliability  must  be  addressed  to  instill 
confidence  in  the  findings.  This  meta-evaluation  relied  on  both  quantitative 
and  qualitative  data.  Conventional  means  of  reliability  and  validity 
assessment  were  used  to  verify  quantitative  data.  To  assess  the  integrity 
of  findings  derived  from  qualitative  methods,  data  collection  processes  and 
findings  were  assessed  for  “trustworthiness” — a term  generally  used  as  a 
substitute  for  “validity”  and  “reliability”  in  qualitative  methodologies.5 

The  following  efforts  were  undertaken  to  ensure  that  the  results  from  the 
meta-evaluation  process  were  trustworthy  and  may  be  interpreted  with 
confidence. 


4 This  monograph  can  be  obtained  from  Alberta  Health  and  Wellness. 

5 Lincoln,  Y.  and  Guba,  E.  (1985).  Naturalistic  inquiry.  Beverly  Hills,  CA:  Sage. 
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Prolonged  Engagement 

The  EMT  was  contracted  by  Alberta  Health  and  Wellness  immediately 
prior  to  the  implementation  of  project.  This  afforded  opportunity  to  work 
with  Project  Directors/Managers  and  independent  evaluators  in  the 
finalization  of  evaluation  plans.  It  also  ensured  congruence  between 
project-level  evaluations  and  provincial  roll-up  of  results.  Through  this 
interaction,  the  EMT  became  very  familiar  with  each  project  from  the 
outset.  Upon  finalization  of  project-level  evaluation  plans,  an  Outcomes 
Framework  was  prepared.  This  Framework  identified  expected  outcomes 
for  all  projects  according  to  the  six  national  dimensions.  The  Framework 
served  as  a useful  reference  point  to  monitor  progress  towards  achieving 
expected  outcomes. 

Credibility  of  Independent  Evaluators 

The  independent  evaluators  were  selected  not  only  on  the  basis  of  their 
expertise  as  evaluators,  but  for  their  experience  in  conducting  evaluations 
in  the  general  area  of  health  care  and/or  in  an  area  specifically  related  to 
their  particular  project. 

Referent  Group 

Critical  discussion  of  evaluation  findings  and  roll-up  by  the  EMT  occurred 
with  members  of  the  Referent  Group.  These  additional  opportunities  to 
reflect  on  policy  issues  provided  further  insights  into  the  impact  of  projects. 
Like  the  EMT,  the  Referent  Group  was  engaged  early  on.  Consequently, 
members  were  very  familiar  with  project  developments. 

Reflective  Sessions  and  Workshops 

Over  the  duration  of  the  Alberta  Primary  Health  Care  Project,  project 
Directors/Managers,  independent  evaluators,  Alberta  Health  and 
Wellness,  the  EMT,  and  other  stakeholders  came  together  for  discussion. 
The  first  provincial  reflective  session  was  held  in  Calgary  in  September 
1998  to  assist  projects  in  refining  project  goals  and  outcomes  and  to 
ensure  that  an  evaluation  plan  was  in  place.  In  October  1999,  a workshop 
was  held  in  Edmonton  where  the  independent  evaluators,  EMT,  Referent 
Group,  and  Alberta  Health  and  Wellness  representatives  discussed  policy 
issues  arising  out  of  the  primary  health  care  projects.  In  April  2000, 
another  Edmonton  workshop  was  held  for  project  Directors/Managers, 
independent  evaluators,  Alberta  Health  and  Wellness  representatives,  and 
the  EMT  to  discuss  the  development  of  plans  to  disseminate  project 
learning.  Finally,  in  July  2000,  Referent  Group  members  met  with  the  EMT 
and  Alberta  Health  and  Wellness  Project  Lead  and  Project  Coordinator  to 
discuss  the  findings  from  the  independent  evaluation  reports.  Through 
these  group  sessions,  primary  health  care  stakeholders  have  had  an 
opportunity  not  only  to  discuss  issues  and  outcomes  from  individual 
projects  but  to  contribute  to  the  overall  discussion  on  what  contribution  the 
Alberta  projects,  individually  and  collectively,  have  made  to  advancing 
primary  health  care. 
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Accountability 

The  results  and  interpretations  contained  in  this  meta-evaluation  final 
report  are  those  of  the  Evaluation  Management  Team  solely.  This  Final 
Independent  Evaluation  Report,  Project  Reports,  and  other  project 
documents  are  available  so  that  others  may  verify  the  analysis  of  the  EMT 
and/or  contribute  other  interpretations  to  this  analysis. 

Presentation  of  Findings 

Several  strategies  were  used  to  explore  and  investigate  primary  health 
care  in  Alberta.  These  strategies  included  (in  no  particular  order): 

■ Rural/Remote  Access; 

■ Prevention  and  Promotion  (Community  Development,  Early 
Intervention  and  Education); 

■ Integrated  Service  Delivery; 

■ Quality  Improvement; 

■ Community  Health  Centre  Models;  and 

■ System  Restructuring. 

A brief  description  of  each  project  follows.  Project  descriptions  are 
categorized  according  to  the  six  strategy  areas  listed  above.  These  six 
areas  comprise  the  first  level  of  analysis — that  is,  grouping  projects 
together  that  were  oriented  to  addressing  primary  health  care  in  a similar 
way.  Grouping  projects  in  this  manner  facilitated  second-  and  third-level 
analyses — the  results  of  which  are  presented  in  the  section  immediately 
following  the  brief  project  descriptions.  These  sections  are  entitled 
“Findings  According  to  the  Six  National  Dimensions  of  Inquiry”  and 
“Implications,”  respectively.  (Refer  to  Appendix  A for  a more 
comprehensive  description  of  the  projects.) 
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Summary  and  Description  of  Strategies  to  Advance 
Primary  Health  Care  in  Alberta 


Rural/Remote  Access 


Rural  Primary  Health  Care  System  Pilot  Project 

This  HA5  project  attempted  to  meet  the  health  needs  of  rural  residents  by 
providing  a mobile  and  multidisciplinary  team  of  health  professionals  to  three 
community  “pods”  on  a weekly  or  semi-weekly  basis:  Standard/Hussar, 
Youngstown/Cessford,  and  Cremona/Water  Valley.  The  study  measured 
rural  community  residents’  self-perceived  health  status,  health  awareness, 
health  attitude  and  behaviour,  perceptions  of  health  services  quality,  and 
accessibility. 

Health  for  All  - Metis  Settlements  Project 

The  Metis  project  involved  the  addition  of  a Settlement  Nurse  to  four  Metis 
settlements  in  the  Lakeland  Regional  Health  Authority.  The  project  was 
intended  to  involve  Metis  community  members  in  identifying  health  needs, 
priorities  and  strategies  to  address  priorities,  and  facilitate  integration  with 
other  parts  of  the  health  system  and  other  health  stakeholders,  while  the 
primary  outcome  was  the  building  of  a trusting  relationship  between  the 
settlements  and  the  Region. 

Telehealth 


This  demonstration  project  evaluated  the  effectiveness  of  telehealth6  in 
providing  enhanced  primary  health  care  services  to  three  northern, 
geographically  remote  communities  (Trout  Lake,  Peerless  Lake,  and  Red 
Earth  Creek)  in  Keeweetinok  Lakes  Regional  Health  Authority. 

Prevention/Promotion  - Community  Development 


Brooks  Cares  Project 


This  project  was  designed  to  address  the  growing  health  needs  of  Brooks’ 
rapidly  expanding  population.  A participatory  approach  was  chosen  as  the 
strategy  to  help  the  citizens  of  this  small  rural  town  identify  issues,  gather 


6 Evaluation  of  the  Usefulness  of  Telehealth  in  Providing  Enhanced  Primary  Health  Services  to  the 
Northern,  Geographically  Remote  Communities  of  Trout  Lake,  Peerless  Lake,  and  Red  Earth  Creek 
(Telehealth). 
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information,  develop  solutions,  and  act  on  solutions.  Training  and 
orientation  sessions  were  offered  to  a small  number  of  community 
members  and  staff.  Some  key  issues  were  identified  by  community 
members,  but  few  newcomers  were  drawn  into  the  project. 

Elnora  Primary  Health  Care  Project 

The  Elnora  demonstration  and  evaluation  project  attempted  to  combine 
primary  health  care  and  community  development  principles  in  the  creation 
of  a rural  Health  Centre  and  the  addition  of  a Nurse  Practitioner  at  the 
Centre.  The  model  was  used  to  provide  Elnora  area  residents  with 
affordable,  accessible,  effective,  and  acceptable  primary  health  care 
services.  The  project  was  developed  as  part  of  a broader  Healthy 
Communities  Initiative  (HCI)  which  involves  five  sites  in  the  David 
Thompson  Health  Region. 

Evaluation  of  Healthy  Okotoks  Project 

This  project  evaluated  the  process  of  creating  a healthy  community 
particularly  with  respect  to  assessing  the  extent  to  which  the  process 
respected  fundamental  values  and  principles  of  health  promotion, 
empowerment,  and  community  development.  It  then  developed  a template 
for  tracking  the  processes  and  outcomes  of  the  Healthy  Okotoks  Coalition, 
a grassroots  health  promotion  initiative,  and  surveyed  Coalition  members' 
satisfaction,  significance  of  changes  made  by  the  Coalition,  and  Coalition 
strength. 

Prevention/Promotion  - Early  Intervention 


Healthy  Families  - Primary  Health  Care  Services  to  High  Risk  Families 
(Capital  Health  Region) 

This  demonstration  project,  based  on  the  Healthy  Families  America 
program,  is  a partnership  between  Capital  Health  and  three  community- 
based  agencies  in  Edmonton.  It  enrolls  families  and  links  them  with  a 
family  visitor  who  makes  weekly  visits  to  provide  education  and  skill 
development  in  parenting  and  to  provide  links  to  community  resources.  A 
community  health  nurse  at  each  community  agency  links  families  to  public 
health  services  and  other  service  providers  in  the  health  system. 

Healthy  Families  Project  (WestView) 

The  WestView  Healthy  Families  Program  is  modeled  after  the  Healthy 
Families  America  Program.  The  program  targets  families  having  their  first 
child  and  provided  intensive  home  visiting  by  a trained  Family  Visitor.  In 
contrast  to  the  Healthy  Families  Capital  project,  the  WestView  project  was 
delivered  through  the  Regional  Health  Authority,  specifically  in  cooperation 
with  public  health  nurses. 
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Keeweetinok  Lakes  Regional  Health  Authority  Immunization  Project 

This  geographically  large,  culturally  diverse,  and  sparsely  populated 
region  is  challenged  by  low  immunization  rates.  The  project  evaluated 
current  immunization  standards  and  interventions  to  populations  at  high 
risk  for  communicable  diseases.  Potential  participants  were  interviewed 
and  invited  to  make  an  appointment  for  immunization.  Characteristics  of 
the  target  population  and  barriers  to  immunization  were  examined  and  the 
effectiveness  and  impact  of  the  intervention  strategy  was  determined. 

Community  Outreach  in  Pediatrics/Psvchiatrv  and  Education  Program 

This  project  investigated  earlier  and  more  accurate  identification  and 
diagnosis  of  children’s  mental  health  problems  through  a school-based 
intervention.  Medical  teams  (physicians  and  pediatricians)  worked  with 
education  teams  (Learning  Specialists,  teachers,  administrators,  and 
Family  Liaisons)  in  four  Calgary  schools.  The  study  also  investigated  the 
effect  of  building  cooperative  relationships  between  medical  and  school- 
based  personnel  and  the  impact  of  the  program  on  the  families  of 
participating  children. 

Prevention/Promotion  - Education 


Misericordia  Health-Lifestyle  Improvement  Education  Centre 

This  demonstration  project  involved  the  creation  of  a Health-Lifestyle 
Improvement  Education  Centre  at  the  Misericordia  Hospital  within  Capital 
Health  Authority.  Physicians  referred  patients  to  the  Adult  Education 
Centre  for  specific  lifestyle  risk  reduction  information  on  cholesterol.  The 
effectiveness  of  this  approach  and  its  impact  on  patients’  health  were 
evaluated. 

Diabetes  Project 

This  project  was  an  evaluation  of  seven  diabetes  centres  in  the  Capital 
Health  Region  according  to  five  guiding  research  questions. 
Characteristics  of  each  Centre’s  population,  accessibility  issues,  level  of 
satisfaction  with  services,  and  knowledge  attainment  after  attending  an 
education  program  were  addressed. 
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Integrated  Service  Delivery 


Lakeland  Integrated  Community-based  Palliative  Care  Project 

This  demonstration  and  evaluation  project  developed  an  integrated  service 
delivery  model  for  palliative  care  services  and  piloted  it  in  the  town  of 
Bonnyville.  The  model  included  a multi-disciplinary  team  of  providers, 
shared  access  to  information,  a procedures  manual,  assessment  tools, 
and  a care  plan  and  communication  passport.  The  goal  of  the  project  was 
to  improve  the  accessibility  and  quality  of  care  palliative  care  services. 

Shared  Mental  Health  Care  In  Primary  Care  Practice 

The  demonstration  project  adopted  a consultation-liaison  service  model  to 
improve  links  between  family  physicians  and  mental  health  professionals 
and  examine  how  mental  health  services  might  be  better  delivered  within 
physician  practices  and  offices.  While  remaining  as  the  primary  caregiver, 
family  physicians  were  supported  by  a multidisciplinary  mental  health  team 
with  the  intention  of  expanding  the  capacity  of  the  family  practitioner  to 
recognize  and  treat  mental  health  issues  in  primary  care. 

Primary  Health  Care  Collectives 

The  goal  of  this  research  project  was  to  develop  and  implement  innovative 
primary  health  care  Collectives  in  which  six  small  groups  of  providers  (a 
family  physician,  a community  pharmacist,  and  a home  care  case 
manager)  developed  and  agreed  on  their  specific  roles,  developed  lines  of 
communication,  and  worked  together  in  a team-based  approach  for  the 
purpose  of  improving  medication  appropriateness  (i.e.,  correct  dosage  and 
use),  patient/client  compliance  with  prescribed  medication  regimens,  and 
medication  management  in  the  community. 

Integrating  the  Services  of  the  Nurse  Practitioner  in  the  Inner  City  (CUPS) 

The  CUPS  (Calgary  Urban  Project  Society)  evaluation  project  described 
the  integration  process  of  a Nurse  Practitioner  (NP)  into  the  collaborative 
practice  primary  health  care  model  that  developed  at  CUPS.  As  well,  the 
project  is  intended  to  service  the  under-served  inner  city  population  of 
downtown  Calgary  in  accordance  with  the  legislation  and  competencies 
set  out  by  the  AARN  for  a registered  nurse  providing  extended  health 
services  in  the  Province  of  Alberta. 

Strengthening  Multidisciplinary  Primary  Health  Care  Teams  in  Coordinated 
Disease  Prevention  and  Management 

This  action  evaluation  and  demonstration  project  adopted  a 
multidisciplinary  team  approach  to  prevention  and  treatment  of  diabetes 
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and  osteoarthritis.  Teams  were  empowered  with  a training  program  and 
internet-based  decision-support  tools  and  followed  a structured  approach 
to  developing  multidisciplinary  care  plans.  Care  plans  were  implemented 
in  six  primary  health  practices,  team  functioning  was  monitored,  and 
effects  on  processes  and  outcomes  were  evaluated. 

Quality  Improvement 

Evaluation  of  Existing  Primary  Mental  Health  Care  Service  at  the 
Edmonton  Centre  for  Survivors  of  Torture  and  Trauma  (ECSTT) 

This  evaluation  project  focused  on  a retrospective  evaluation  of  the 
Community  Strengthening  Program,  specifically  the  Early  Intervention 
Program  which  focused  on  non-therapeutic  aspects  of  the  program 
serving  children  in  three  cultural  communities:  Somalian,  Cambodian,  and 
Bosnian.  As  well,  the  evaluation  focused  on  assessing  client  satisfaction 
and  success  of  the  Somali  Women’s  Program.  Evaluation  also  included  an 
assessment  of  the  clinical  program.  The  evaluation  process  involved 
ECSTT  staff  and  volunteers,  and  Somali  women  and  children  who  had 
previously  attended  the  programs. 

Evaluation  of  Urban  Patients’  Choice  of  an  Emergency  Department  as 
Their  First  Contact  with  Primary  Health  Care  Services 

This  research/evaluation  project  was  conducted  at  the  Peter  Lougheed 
Centre  (PLC)  within  the  Calgary  Regional  Health  Authority.  It  investigated 
why  people  used  the  emergency  department  for  primary  health  care  needs 
and  what  knowledge  users  had  of  alternate  services.  The  demographic 
composition  of  the  multicultural  catchment  population,  profile  of  patients 
using  the  Emergency  Department  for  primary  care,  access  patterns,  and 
barriers  to  access  were  described  and  assessed. 

Evaluation  of  a Primary  Health  Care  Clinic  According  to  the  Primary 
Health  Care  Parameters  of  First  Contact,  Longitudinally, 
Comprehensiveness  and  Coordination  - UCMC  Sunridqe  Project 

This  project  was  to  create  a general  methodology  for  evaluating  a family 
practice  medical  clinic.  The  project  aimed  to  develop  a process  for  the 
measurement  of  the  dimensions  of  primary  care  (first  contact/access, 
longitudinally,  comprehensiveness,  and  coordination),  review  asthma  care 
as  an  indicator  of  quality  of  care,  and  identify  key  success  factors  for 
strategic  development  of  a primary  health  care  evaluation  model  for  a 
community-based  family  practice  clinic  in  Calgary. 
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Enhancing  Palliative  Care  for  Cancer  Patients  (Calgary  Regional  Health 
Authority) 

This  demonstration  project  developed  strategies  to  improve  the  quality  of 
care  for  palliative  cancer  patients.  It  (a)  created  and  evaluated  a Quality 
Improvement  Process  to  develop  and  apply  criteria  for  the  selection  of 
projects;  (b)  assessed  the  needs  of  family  physicians,  Community  Care 
Coordinators,  oncologists,  patients,  and  families;  (c)  developed, 
disseminated,  and  evaluated  a Palliative  Care  Brochure;  (d)  developed, 
implemented,  and  evaluated  a standardized  consultation  note  format;  and 
(e)  developed,  implemented,  and  evaluated  a Clinical  Practice  Guideline 
for  Palliative  Sedation. 

Community  Health  Centre  Models 

Airdrie  Regional  Health  Centre 

The  goal  of  the  Airdrie  evaluation  project  was  to  monitor  the 
implementation  process  and  the  first  year  of  operation  of  the  Airdrie 
Regional  Health  Centre,  a collaborative  project  by  the  Calgary  Regional 
Health  Authority,  Rocky  View  Municipal  District  No.44,  and  the  City  of 
Airdrie.  In  collaboration  with  other  community  organizations,  a multi-use 
community  health  facility  was  conceptualized  and  a wide  range  of  health 
services  offered. 

8th  and  8th  Health  Centre 

The  purpose  of  the  evaluation  project  was  to  review  operations  of  the  8th 
and  8th  Health  Centre  in  downtown  Calgary  (since  its  inception  March 
1997)  with  a view  to  identifying  strengths,  opportunities  for  improvement, 
what  has  been  learned  to  date,  and  to  make  recommendations  to  support 
ongoing  development  of  the  Centre.  The  intent  was  to  facilitate  decision 
making  regarding  future  services  at  8th  and  8th  and  to  explore  the  potential 
implications  of  subsequently  developing  other  similar  Health  Centres 
within  the  CRHA. 

Northeast  Community  Health  Centre 

This  Edmonton  demonstration  and  evaluation  project  deployed  a 
multidisciplinary  “enhanced  service  team”  (dietician,  occupational 
therapists,  addictions  counselor,  mental  health  therapist,  social  worker, 
and  multicultural  health  broker)  to  meet  the  diverse  needs  of  the  Northeast 
community  in  concert  with  a primary  health  services  team  (family 
physicians,  nurse  practitioner,  nurses,  and  other  health  professionals). 
During  the  first  18  months  of  operation  of  this  CHC,  implementation  and 
formative  evaluations  were  conducted,  and  the  enhanced  service  team 
was  evaluated. 
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Evaluation  of  the  Alexandra  Community  Health  Centre 

The  Alexandra  Community  Health  Centre  in  Calgary  operates  programs  in 
three  areas:  clinical  service,  psychological  service,  and  community 
development.  This  internal  evaluation  project  was  to  examine  the 
Alexandra  CHC  as  a primary  health  care  organization  with  respect  to  four 
dimensions:  Client  Description,  Goals,  Process,  and  Impact. 

System  Restructuring 


East  Central  Health  Primary  Health  Services  Initiative 

This  demonstration  project  intended  to  establish  integrated  primary  health 
services  in  four  rural  communities  and  surrounding  areas  in  the  East 
Central  Regional  Health  Authority.  The  intent  was  to  catalyze  and  support 
the  evolution  of  integrated  primary  health  services  in  the  four  rural 
communities  (Consort,  Coronation,  Hardisty  and  Castor)  and  eventually 
across  the  region.  The  initiative  focused  on  working  with  communities  in 
partnership  to  build  an  integrated  sustainable  primary  health  system  in 
rural  communities. 

A Study  to  Support  the  Repositioning  of  Services  and  Practices  in  the 
Provision  of  Primary  Health  Care  (Calgary  Regional  Health  Authority) 

This  demonstration  project  used  a "whole  systems"  model  to  examine  the 
existing  health  system  in  Calgary  Regional  Health  Authority.  Over  600 
primary  health  care  services  were  mapped  to  create  a visual 
representation  of  services,  providers,  and  populations  with  the  Region. 
Game  theory  was  used  to  involve  stakeholders  in  developing  a model  for  a 
new  health  system  that  would  optimize  health  and  use  resources 
efficiently.  Some  analysis  of  the  new  model  was  conducted  with  groups  at 
three  primary  care  sites  and  barriers  to  piloting  the  new  model  were 
described. 
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Access 


The  dimension  of  access  was  oriented  toward  investigating  changes  to 
access  to  available  health  services  or  preferred  providers;  changes  in 
waiting  times  and  the  provision  of  culturally  and  linguistically  sensitive 
services;  changes  in  the  distribution  of  access  by  gender,  geographic 
location  and/or  ethnic  group;  and  changes  in  location  or  accommodations 
where  services  are  provided  (e.g.,  co-location).  Projects  in  Alberta  tended 
to  focus  on  three  aspects  of  access: 

a)  assessment  of  the  need  for  service, 

b)  development  of  indicators  for  assessing  access,  and 

c)  investigation  of  various  strategies  to  address  barriers  to  access. 

Needs  Assessment 


Tension  between  Want  and  Need 


Typically,  access  is  measured  as  a function  of  utilization  of  service.  Within 
a primary  health  care  context,  however,  determination  of  appropriate  or 
sufficient  access  depends  on  a negotiation  between  competing 
assumptions  and  perceptions  about  what  is  considered  need  or 
“necessary”  health  services.  The  negotiation  between  what  a community 
desires  and  what  the  health  system  is  willing  to  provide  may  lead  to 
tension  and  delays  in  planning  processes.  This  was  the  experience  of  one 
of  the  primary  health  care  projects. 

This  tension  became  evident  between  what  community  members  regarded 
as  health  needs  and  those  the  region  was  willing  to  support.  This  situation 
negatively  influenced  the  expeditious  development  of  a service  delivery 
model  for  the  Airdrie  Regional  Health  Centre,  which  was  regarded  as  an 
essential  starting  point  from  which  to  reflect  evaluation  results  of  the  first 
year  of  operation  of  the  Centre. 

A holistic  primary  health  care  approach  was  used  to  plan  the  original 
Health  Centre,  and  the  unique  service  needs  of  the  City  of  Airdrie  had 
been  considered.  However,  once  it  became  evident  that  public  funding 
would  not  support  the  vision  of  an  innovative,  comprehensive,  multi-use 
Health  Centre  (which  was  the  community’s  vision),  the  focus  of  planning 
shifted  solely  to  planning  a physical  facility  to  replace  the  inadequate 
public  health  site,  with  provision  for  community  agencies  as  facility  tenants 
(i.e.,  the  Food  Bank,  Family  Services,  and  a resource  centre  for  persons 
with  disabilities). 
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It  is  not  surprising  that  at  the  end  of  the  two-year  project,  evaluation 
findings  indicated  continued  strong  public  expectation  and  demand  for 
access  to  medical  services  at  the  Health  Centre — an  expectation  the 
region  (CRHA)  was  unable  to  dispel. 


Although  early  planning  processes  reflected  open  and  transparent  discussions 
with  community  leaders,  and  planning  forums  readily  encouraged  public 
involvement,  such  communication  was  greatly  curtailed  once  the  after  hours 
controversy  emerged  and  failed  to  subside  (Wanke,  2000). 


To  the  public,  the  term  “Health  Centre”  meant  medical  and  support 
services,  including  laboratory  and  medical  imaging.  Community  perception 
of  need  for  these  enhanced  medical  services  did  not  match  regional 
commitment — particularly  when  the  region  itself  was  undergoing 
organizational  restructuring.  In  the  context  of  this  change  the  project 
struggled  to  move  forward.  In  the  end,  the  Project  Director,  in  consultation 
with  the  Centre  development  team,  was  able  to  produce  an  Airdrie 
Regional  Health  Centre  Service  Delivery  Model  which  serves  as  a 
framework  for  future  evaluations  of  the  Health  Centre.  The  Model  makes 
explicit  the  key  elements  of  service  delivery,  specifically  the  “client  needs” 
component.  Client  need  serves  as  the  denominator  for  deciding  the  extent 
to  which  services  are  delivered  equitably  or  reasonably  to  community 
members — in  other  words,  for  assessing  access. 

Figure  1.  Airdrie  Regional  Health  Centre  Service  Delivery  Model 
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A somewhat  different  experience  between  community  and  region  resulted 
in  the  development  of  a community  capacity  approach  to  identify  need  for 
health  service.  This  assessment  of  access  involved  a planning  process 
held  at  the  regional  level  (East  Central)  to  coordinate  services  in  four  rural 
communities  (and  eventually  across  the  region).  Over  the  course  of  the 
project,  the  orientation  to  access  became  increasingly  qualified  by  a 
concern  that  service  gaps  be  filled,  that  duplication  be  eliminated,  and  that 
access  issues  in  the  four  communities  be  considered  in  a way  that 
included  neighboring  towns,  villages,  and  hamlets.  The  focus  on  health  at 
the  community  level  resulted  in  a working  definition  of  access  that 
considered  community  capacity  for  health.  Capacity  included  the  ability  to 
respond  to  crises,  deal  with  important  issues  (e.g.,  physician  recruitment) 
and  a range  of  health  services  and  supports.  One  might  consider 
community  capacity  as  influencing  the  perception  of  access  in  that  once 
community  members  are  aware  of  and  involved  in  decisions  related  to  the 
identification  of  “necessary”  health  services,  their  judgments  of  accessible 
health  services  are  likely  to  depend  on  the  extent  to  which  they  believe 
services  are  available  to  meet  the  needs  they  identify.  The  project 
engaged  community  members  in  setting  the  needs  assessment  process  in 
motion. 


A Total  Health  Process 

Another  attempt  to  assess  access  to  health  service  at  the  regional  level 
involved  the  identification  of  gaps  in  service  delivery  through  the  use  of  a 
Total  Health  Process.  This  process  involved  mapping  and  matching  of 
population  needs  to  resource  allocation  through  a population  modeling 
exercise  (Repositioning  project).  Independent  evaluators  regarded  these 
processes  as  holding  potential  to  improve  access  to  services  delivered. 
For  example,  those  patients  identified  as  being  under  served  would  likely 
realize  improved  access  to  services. 

If  a model  such  as  the  Total  Health  Process  were  to  function  cost-effectively,  the 
behavior  of  both  providers  and  patients  would  need  to  be  regulated  (Church, 
Gerlock,  La  Fleur,  & Mark,  2000). 


However,  the  evaluators  cautioned  that  under  capitated  arrangements, 
providers  run  the  risk  of  budget  overruns  where  patients  do  not  commit  to 
receive  services  from  designated  physicians.  Another  drawback  is  a 
possible  tendency  in  managed  care  arrangements  employing  capitated 
budgets  for  providers  to  seek  out  those  patients  who  will  use  fewer 
resources.  The  independent  evaluators  suggested  that  significant  changes 
will  be  required  to  current  legislation  in  Canada  to  successfully  implement 
such  a Total  Health  Process. 

Assessment  of  Access  at  the  Facility  Level 

Three  projects  focused  on  assessing  access  to  health  service  at  the 
facility  level.  In  one  case,  assessment  indicated  that  the  demographics  of 
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the  target  population  had  changed  significantly  since  the  opening  of  the 
Alexandra  Community  Health  Centre  almost  thirty  years  prior  to  the  HTF 
project.  When  the  Centre  was  first  established  it  was  to  assist  low-income, 
ethnic-based  populations  in  three  inner-city  neighbourhoods  in  Calgary.  At 
least  two  of  these  neighbourhoods  have  undergone  significant  socio- 
economic changes,  and  the  current  population  of  these  neighbourhoods  is 
less  likely  to  require  the  services  of  the  Alexandra  Centre  than  in  earlier 
times.  Evaluation  findings  revealed  that  the  Centre  is  no  longer  well 
integrated  into  the  community.  The  Board  of  the  Centre  has  now  redefined 
its  target  population  and  refocused  the  services  it  delivers.  For  example, 
plans  are  underway  to  establish  satellite  clinics  to  better  meet  clients’ 
needs.  Evening  clinics  are  also  being  considered. 

At  a second  community  health  centre,  the  Northeast  Community  Health 
Centre  in  Edmonton,  decisions  about  service  delivery  have  been  based  on 
extensive  community  consultations  as  well  as  on  a review  of 
epidemiological  evidence  (regional  status  reports).  To  ensure  that  services 
meet  the  needs  of  their  target  population,  50%  of  practice  time  within  the 
Family  Health  Department  has  been  reserved  to  serve  the  target  group. 
The  opening  of  the  Centre,  whose  primary  aim  was  to  provide 
neighbourhood-based  services,  also  aimed  at  reducing  barriers  to  access 
by  clients.  To  that  end,  vouchers  have  been  made  available  for  some 
clients  to  address  transportation  difficulties.  Because  access  to  services  is 
an  integral  feature  of  the  Centre,  ongoing  discussion  about  access  issues 
occurs  between  Centre  staff  and  the  Community  Consultation  Committee 
which  was  established  to  represent  community  members. 

In  the  third  project  addressing  the  access  dimension,  issues  of  access 
were  explored  in  relation  to  the  Emergency  Department  (ED)  at  a tertiary 
care  health  centre.  Baseline  data  were  collected  on  the  multicultural 
population  of  the  Peter  Lougheed  Centre  (Calgary)  as  a way  of 
determining  profiles  on  clients  who  typically  access  the  ED  for  primary 
health  care  purposes. 

Findings  suggest  that  patients  make  reasonable  choices  about  accessing 
primary  health  services  at  the  ED.  For  example,  patients  that  participated 
in  the  study  reported  that  they  would  attend  the  ED  for  increasingly  urgent 
situations.  (Actual  usage  patterns  supported  this  finding.)  More  than  twice 
as  many  patients  with  more  resource-intense  conditions  attempted  to 
contact  their  family  physician  prior  to  going  to  the  ED  than  patients  with 
less  resource-intense  conditions.  Most  patients  presenting  to  the  ED  were 
not  regular  users  of  ED  for  primary  care  (also  supported  by  utilization 
data).  A significant  proportion  of  users  reported  that  they  had  attempted  to 
contact  another  medical  professional  prior  to  going  to  the  ED  (a  family 
physician  or  a walk-in  clinic).  Many  of  these  users  had  also  consulted  a 
non-professional,  such  as  a family  member. 
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Despite  having  a family  physician,  as  82%  of  the  study  participants  did, 
nearly  half  of  this  group  felt  they  had  access  to  care  elsewhere  but  chose 
the  ED  instead.  Interestingly,  proximity  to  the  ED  did  not  appear  to 
influence  visits  to  the  ED.  Convenience  was  rarely  cited  as  a reason  to 
visit  the  ED.  Neither  was  wanting  to  see  a specialist.  Also,  very  few  study 
participants  had  been  directed  to  the  ED  by  a physician’s  office. 

It  is  also  interesting  to  note  that  the  population  using  the  Peter  Lougheed 
Centre  was  less  culturally  diverse  than  expected.  Even  though  the  Centre 
is  located  in  a multicultural  area  of  the  City  of  Calgary,  96%  of  study 
participants  spoke  English  (although  the  evaluator  noted  that  Vietnamese, 
Chinese,  and  Aboriginal  groups  may  have  been  under-surveyed  in  the 
study). 


Findings  indicated  that  worry  and  sense  of  urgency  drove  the  majority  of  patients 
to  the  ED  for  primary  care  service,  despite  access  to  other  options  (D’ Andrea, 
2000). 


Addressing  Barriers 

Several  projects  have  contributed  information  about  strategies  for 
addressing  barriers  to  accessing  health  services.  Strategies  included  the 
introduction  of  a Nurse  Practitioner,  a Settlement  Nurse,  a community 
coalition,  community-based  mental  health  service,  school-based  mental 
health  service,  community-based  palliative  care  service,  mobile  service 
teams,  and  collectives  of  health  professionals.  Each  strategy  and  its  result 
is  addressed  below. 

Nurse  Practitioner  Strategy 

The  introduction  of  the  Nurse  Practitioner  (NP)  at  an  inner  city  health 
centre  (Calgary  Urban  Project  Society — CUPS)  resulted  in  a decrease  in 
the  number  of  clients  turned  away  (an  overall  reduced  turn-away  rate  of 
40%).  The  addition  of  the  NP  also  expanded  the  levels  of  care  available 
(i.e.,  M.D.,  RN,  NP,  and  LPN).  As  well,  evaluation  findings  suggested  that 
clients  saw  the  appropriate  provider  more  frequently  because  the  NP  was 
able  to  deal  with  clients  who  did  not  require  the  attention  of  a physician 
(e.g.,  drug  prescription  clients  from  the  8th  and  8th  Health  Centre). 

However,  while  staff  at  CUPS  held  a common  perception  that  the 
integration  of  the  NP  role  into  the  primary  health  team  at  CUPS  had 
improved  access  to  care  by  CUPS  clientele,  evaluation  findings  indicated 
that  the  total  number  of  clients  seen  over  the  two  years  of  the  project  had 
remained  relatively  stable. 

Some  concern  was  raised  by  both  physicians  and  clients  of  the  reduced 
ability  of  clients  to  see  the  same  physician  at  each  visit  since  some  clients 
were  seen  by  the  NP  instead  of  the  physician.  However,  client  satisfaction 
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results  did  not  indicate  dissatisfaction  with  having  to  see  the  NP. 
Introduction  of  the  NP  also  resulted  in  increased  access  to  health  service 
for  hard-to-reach  groups.  The  NP  held  an  outreach  clinic  at  a neighboring 
drop-in  centre  (Calgary  Drop-In  Centre). 

Introduction  of  a Nurse  Practitioner  at  a second  site,  a rural  health  centre 
(Elnora),  resulted  in  community  awareness  and  subsequent  utilization  of 
available  services  increasing  over  the  course  of  the  project.  Community 
members  had  little  difficulty  understanding  the  difference  in  services 
provided  by  the  NP  as  opposed  to  the  visiting  physician.  In  fact,  the 
internal  evaluator  suggested  that  community  members  helped  to  define 
the  NP’s  role  through  the  different  services  they  asked  her  to  perform.  A 
community  survey  indicated  satisfaction  with  NP  service.  This  did  not, 
however,  seem  to  influence  community  perception  of  the  need,  or  want,  of 
a physician  in  their  rural  community.  This  latter  issue  continues  to  fester.  A 
subsequent  initiative  in  the  region  includes  plans  to  move  the  NP  to 
another  community,  in  which  case  community  members’  perceptions  of 
access  to  service  at  the  Elnora  Health  Centre  may  change  markedly. 

A Settlement  Nurse  Strategy 

Introduction  of  a Settlement  Nurse  on  four  Metis  Settlements  in  Lakeland 
Regional  Health  Authority  resulted  in  some  interesting  findings.  Public 
health  and  home  care  nurses  did  not  see  any  improvement  in  the 
knowledge  of  Settlement  residents  about  public  health  and  home  care 
services  available  in  the  region.  However,  Settlement  residents  who 
responded  to  the  survey  commented  frequently  on  the  ease  of  access  to 
services  provided  on  Settlement  by  the  Settlement  Nurse. 


Settlement  residents  recognized  that  “their”  nurse  (the  Settlement  Nurse)  had 
information  that  could  link  them  to  other  health  services  if  they  needed  them 
(Matthias,  2000). 


By  living  on  the  Settlement  with  Settlement  residents,  the  Settlement 
Nurse  was  able  to  remove  attitudinal  and  knowledge  barriers  about  health 
services — barriers  Settlement  residents  may  have  perceived  to  be 
geographic  barriers  to  health  services  located  in  nearby  off-Settlement 
locations.  Access  to  health  service  then,  was  facilitated  through  building  a 
relationship  between  the  Settlement  Nurse  and  Settlement  residents. 

What  is  more,  as  a result  of  this  relationship  the  Nurse  was  able  to  identify 
groups  with  common  health  needs.  This  opportunity  acted  as  a precursor 
to  providing  appropriate  health  services  based  on  identified  community 
need. 

Co-location  Strategy 


While  the  Airdrie  Community  Health  Centre  was  experiencing  some 
growing  pains,  some  positive  indicators  of  improved  access  surfaced.  Co- 
location  of  health  services  within  a community  Health  Centre  with  Family 
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Services,  the  Food  Bank,  and  an  information/resource  centre  for  persons 
with  special  needs  and  disabilities  resulted  in  enhanced  access  to  the 
Health  Centre  programs.  Findings  indicated  that  improved  access  may 
have  been  due  to  the  central  and  visible  location  of  the  Centre  on  the  city’s 
main  thoroughfare,  and  close  proximity  to  the  major  shopping  area  and 
large  free  parking  area.  Increased  walk-in  traffic  was  reported.  Co-location 
of  CRHA  staff  with  partner  agencies  also  resulted  in  easier  referral 
processes  and  greater  numbers  of  referrals  to  Centre  programs.  The 
availability  of  community  meeting  facilities  led  to  enhanced  access  to 
educational  sessions  and  presentations  by  individual  clients  and 
community  groups.  So,  while  Centre  staff  would  have  preferred  that 
services  were  integrated,  they  were  at  least  co-located. 

Mobile  Health  Service  Strategy 

The  addition  of  mobile  health  teams  to  four  rural  communities  in  Alberta 
(Health  Authority  5)  helped  to  reduce  geographic  barriers  to  service. 
Evaluation  results  indicated  that  young  children  and  seniors  accessed 
mobile  services  more  frequently  than  did  other  groups  in  the  four  rural 
communities.  Overall,  the  number  of  referrals  to  the  clinics  set  up  in  the 
rural  communities  began  to  increase  over  the  course  of  the  project.  In  fact, 
new  referral  patterns  were  beginning  to  emerge  at  project  close. 

One  of  the  aims  of  the  project  was  to  have  fewer  people  leave  the  region 
to  access  physicians'  services  since  residents  often  traveled  to  Calgary  to 
access  health  services.  This  was  accomplished  for  family  physician  visits. 
A decrease  in  the  number  of  people  accessing  primary  health  services  in 
Calgary  was  reported.  However,  an  increase  in  the  number  of  people 
traveling  to  Calgary  to  access  specialist  services  was  also  reported.  A 
higher  proportion  of  the  population  was  seeking  health  information  or 
reported  receiving  health  information.  These  findings  suggest  that  when 
people  are  aware  of  services  available  in  their  home  communities,  they 
are  likely  to  access  them  (at  least  in  rural  Alberta). 

Community-based,  Integrated  Health  Service  Delivery 

Another  project  addressed  geographic  barriers  to  health  service — 
specifically  palliative  care  service.  The  Lakeland  Palliative  project 
addressed  the  exodus  of  patients  to  the  nearby  Capital  Health  Authority 
for  palliative  cancer  treatment.  Two  key  strategies  were  used  to  improve 
access  to  palliative  care  service  in  Lakeland  region.  These  included  a 
central  client  registry,  referral,  intake  and  assessment  process,  which 
eliminated  patients  having  to  repeat  their  story  to  multiple  providers.  At  the 
same  time,  loss  of  information  between  levels  of  care  was  avoided.  A 
common  Communication  Passport  allowed  both  providers  and  patients  to 
monitor  care  plans.  However,  the  strategy  that  had  the  most  significant 
impact  on  satisfaction  levels  of  families/caregivers  was  24-hour  access  to 
the  Palliative  Nurse  Consultant  and  home  care  nurses.  In  dealing  with 
issues  of  pain  management,  access  to  consultation  advice  from  a health 
professional  was  considered  critical  to  family  members. 
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“. . .she  wanted  to  be  pain  free  and  at  home.  The  Palliative  Care  program  helped 
us  achieve  this  goal.  You  couldn’t  have  asked  for  anything  better  (Lakeland 
Palliative  Care  Project  Report,  2000). 


Community-based  Mental  Health  Strategies 

Three  types  of  community-based  mental  health  strategies  were  used  to 
increase  access  to  mental  health  services  for  adults,  school  children,  and 
newcomers  to  Canada.  The  shared  care  strategy  (which  involved 
psychiatrists  and  counsellors  consulting  with  family  physicians  in  their 
offices)  was  successful  in  increasing  community  members’  convenient 
access  to  mental  health  service.  (Specific  impact  on  client  health  status  is 
addressed  under  the  Health  Status  dimension.)  This  strategy  helped  to 
remove  the  stigma  of  visiting  psychiatrists’  offices  or  other  facilities 
dedicated  to  mental  health  treatment. 

A school-based  mental  health  program  (C.O.P.E.)  provided  increased 
access  to  community-based  diagnostic  and  consultation  service  by 
medical  team  members  in  schools.  This  strategy  was  successful  in  timely 
diagnosis  of  children’s  emotional,  behavioural,  and  learning  problems. 
Where  required,  a Family  Liaison  provided  transportation  to  consultation 
appointments. 


The  C.O.P.E.  strategy  was  successful  in  reducing  waiting  times  from 
approximately  one  year  to  two  weeks  (Howard,  2000). 


A third  community-based  strategy  involved  providing  mental  health 
services  to  refugees,  immigrants,  and  other  multicultural  groups  of 
newcomers  through  on-site  counselors  and  therapists  at  the  Edmonton 
Centre  for  Survivors  of  Torture  and  Trauma  (ECSTT).  There  was  some 
indication  that  clients  accessed  services  more  readily  because  of  their 
centralized  location  at  the  ECSTT  and  because  children's  programs  ran 
simultaneously  with  one  of  the  programs  offered  to  adults  (i.e.,  the  Somali 
Women's  Program).  Scheduling  program  and  service  offerings  in  this 
manner  eliminated  the  need  for  childcare,  thereby  also  freeing  women 
(and/or  men)  to  access  on-site  counseling  services. 

I “Many  of  the  women  got  jobs  through  the  Mennonite  Centre  got  their  resumes 
done  and  were  taught  how  to  get  jobs  through  the  employment  area.  Just  being 
part  of  the  program  they  were  able  to  access  other  services  within  the  Mennonite 
Centre”— Staff  member  (McCoy,  2000). 
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Addressing  Information  Barriers 
Information  about  Palliative  Care 

An  interesting  finding  arose  in  the  dissemination  of  an  educational 
brochure  to  palliative  patients/families  in  the  Calgary  Palliative  Care 
project.  While  only  one  portion  of  a four-part  initiative  to  improve  the 
quality  of  care  to  palliative  patients,  learning  from  project  efforts  to  share 
information  about  community-based  palliative  care  resources  within  the 
Calgary  region  is  useful  to  others. 

Although  the  development  of  the  educational  brochure  had  gone  through  a 
comprehensive  stakeholder  consultation  and  development  process,  and  a 
sizable  distribution,  only  a small  number  of  palliative  patients  and  their 
families  reported  having  seen  the  brochure.  This  raised  concern  among 
the  Leadership  Team  about  the  viability  of  the  brochure  and  also  the 
readiness  of  patients  and  their  families  to  accept  specific  kinds  of 
information  at  certain  stages  of  their  illness.  The  Leadership  Team 
concluded  that  creative  ways  needed  to  be  developed  to  attract  people’s 
attention — particularly  when  patients  and  families  are  struggling  with  the 
tension  of  assimilating  a terminal  prognosis  with  not  wanting  to  lose  hope. 
Print-based  media  may  be  effective  when  used  in  combination  with  other 
innovative  presentation  formats  and  techniques. 

Information  about  Medication  Use  and  Management 

The  Collectives  project  found  that  improved  access  to  and  coordination  of 
pharmacy  and  home  care  services  with  family  practice  services  for  people 
with  complex  health  needs  helped  to  significantly  improve  clients’ 
medication  compliance.  Increased  access  to  information  by  clients  led  to 
improved  self-managed  care.  Approval  by  the  Alberta  College  of 
Physicians  and  Surgeons  to  have  any  member  of  the  Collective  enroll 
patients  was  also  regarded  as  a factor  that  increased  client  access  to  the 
program.  (Approval  has  not  yet  been  granted  by  the  Alberta  Medical 
Association.) 
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Addressing  Attitudinal/Knowledqe/Cultural  Barriers 


Building  Capacity  to  Identify  and  Act  on  Community  Need 

The  Healthy  Okotoks  Coalition  facilitated  access  to  activities  that 
promoted  healthy  living  for  community  members.  Some  of  these  activities 
included 

(a)  the  creation  of  a Community  Garden, 

(b)  an  annual  Fall  clean-up, 

(c)  FAS  awareness  and  healthy  babies  pilot  project, 

(d)  increased  awareness  of  FAS  among  8th  graders, 

(e)  lead  free  paint  for  playground  equipment, 

(f)  youth  curfew  implemented  for  youth  under  15  years  old, 

(g)  formation  of  a youth  action  group, 

(h)  alteration  of  the  roller-blade  bylaw,  and 

(i)  skateboard  park  approval  by  Town  Council. 

The  Coalition  found  that  involving  community  members  in  a non- 
threatening way  through  activities  that  were  informal  and  geared  to 
physical  activity,  particularly  through  children  and  youth,  appeared  to  have 
potential  to  reach  marginalized  groups.  The  Coalition,  then,  was  a 
demonstration  of  a successful  community  development  approach  to 
improve  access  to  health-enhancing  opportunities. 

Establishing  Relationship  with  At-Risk  Mothers 

Two  primary  health  care  projects  focused  on  extending  direct  personal 
contact  to  at-risk  women.  Capital  Health  Authority’s  Healthy  Families 
project  was  able  to  reach  women  through  existing  agency  networks  that 
had  already  been  established  to  work  with  at-risk  clients  (e.g.,  agencies 
dealing  with  unwed  mothers,  disadvantaged  Aboriginal  women).  Referrals 
to  the  program  were  facilitated  through  these  agencies  and  also  through 
public  health  nurses  and  self-referral.  A Liaison  Worker  was  linked  to  the 
Child  and  Family  Services  Authority  so  that  referrals  were  also 
forthcoming  from  cases  where  it  was  considered  that  children/families 
could  benefit  from  the  healthy  families  early  intervention.  Multi-cultural 
health  brokers  also  facilitated  access  to  health  services  for  this  group. 
Approximately  400  families  were  screened,  almost  350  referred,  and 
approximately  250  enrolled  in  the  program. 

In  contrast,  a total  of  45  families  were  enrolled  in  the  Healthy  Families 
program  in  the  WestView  region.  Here  the  point  of  referral  was  through  the 
hospital  discharge  system.  In  both  regions,  the  Family  Visitor  played  an 
important  role — in  WestView  helping  families  to  link  to  other  services  (e.g., 
the  majority  of  clients  had  attended  Well  Baby  Clinics  at  the  Health  Unit 
and  accessed  at  least  one  other  community  agency).  While  the  Family 
Visitor  in  Capital  worked  through  agency  staff,  in  WestView  the  Family 
Visitor  worked  through  the  public  health  nurse  (the  existing  regional  health 
structure).  It  appears  that  the  agency-based  model  may  have  greater 
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potential  for  clients  to  be  aware  of  the  support  available  to  them  through 
the  Healthy  Families  program.  Follow-up  interviews  with  WestView  clients 
indicated  that  72%  of  families  who  refused  the  program  could  not  recall  its 
general  purpose.  Independent  evaluators  suggested  that  a social 
marketing  campaign  may  be  necessary  to  raise  awareness  of  the 
program. 

Right  of  Refusal  Strategy  for  Increased  Immunization 

The  influence  of  making  personal  contact  with  clients  on  access  to  health 
services  was  also  evident  in  the  Immunization  project,  which  explored  a 
Right  of  Refusal  strategy  for  parents/guardians  choosing  not  to  have  their 
children  immunized.  In  this  rural  health  region,  clients  who  did  not  maintain 
the  immunization  status  of  their  children  appeared  to  have  less  knowledge 
about  the  need  and  importance  of  immunization  in  general,  and  less 
awareness  of  their  child’s  immunization  schedule.  Primary  caregivers  of 
the  under-immunized  were  typically  single,  Aboriginal,  younger  in  age,  and 
had  a lower  level  of  education  and  income.  Statistically,  clinic 
inaccessibility  proved  to  be  the  highest  predictor  among  barriers  to 
immunization.  Other  significant  barriers  cited  by  the  under-immunized 
group  were  lack  of  childcare  and  transportation  problems.  Findings  also 
suggested  that  written  refusal  did  not  appear  to  be  an  effective  strategy  for 
dealing  with  under-immunization.  (The  Right  of  Refusal  was  only 
applicable  and  offered  to  1 1 people.)  The  project  team  attributed  an 
increase  in  immunization  rates  of  the  under-immunized  group  over  the 
course  of  the  project  largely  to  the  telephone  call  from  the  researcher. 
Personal  contact,  the  team  surmised,  was  a critical  factor  in  increasing 
immunization  rates  in  the  region. 

Financial  Disincentives 


The  present  fee  for  service  structure  in  Alberta  creates  a financial 
disincentive  for  health  professionals  to  engage  in  collaborative 
consultation  opportunities.  For  example,  physicians  are  not  remunerated 
for  consulting  with  other  medical  professionals  without  the  patient  being 
present.  The  C.O.P.E.  project  highlighted  that  while  psychiatrists  could  bill 
for  consultation  time  without  the  patient  being  present,  pediatricians  could 
not.  This  disparity  created  a barrier  to  collaborative  activity  among  medical 
professionals. 

The  Collectives  project  encountered  a similar  difficulty. 


“Community  pharmacists  are  paid  through  individual  drug  dispensing  fees  set  by 
Alberta  Health  and  Wellness,  based  on  the  cost  of  the  drug,  on  a fee-for  service 
basis.  They  are  not  paid  directly  for  what  is  known  as  “cognitive  services”  such  as 
consultation  with  patients/clients  or  with  other  health  care  professionals” 

(Gardner,  2000).  


Theoretically,  the  Lifestyle  Risk  Reduction  program  (specifically  the 
Education  Centre  component)  improved  access  to  continuing  and  holistic 
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care  for  clients  with  elevated  serum  cholesterol  beyond  treatment  offered 
in  the  traditional  primary  care  office  setting.  There  appears  to  be  little 
incentive  under  the  current  fee-for-service  system  for  family  physicians  to 
spend  extra  time  with  patients  with  elevated  cholesterol.  This  program 
provided  an  alternative.  If  the  program  continues,  there  is  a waiting  list  of 
clients  willing  to  choose  a diet  and  exercise  program  over  medication. 

Indicators  of  Access 

A set  of  eight  indicators  was  developed  as  part  of  a Continuous  Quality 
Improvement  evaluation  template  for  a family  practice  medical  clinic 
(UCMC-Sundridge).  Indicators  assess  access  to  service  through  wait  time, 
availability,  and  utilization  of  services,  specifically: 

(a)  lag  time  in  getting  appointment  as  a function  of  severity  of  problem, 

(b)  clinic  hours, 

(c)  availability  of  emergency  times, 

(d)  provisions  of  after-hour  coverage, 

(e)  availability  of  after-hours  access, 

(f)  patients  likelihood  of  using  clinic  if  medical  problem  arises, 

(g)  patients  alternative  choice  when  clinic  is  closed,  and 

(h)  consultation  length. 

These  indicators  now  serve  as  a component  of  the  overall  evaluation 
approach  to  determining  the  quality  of  services  provided  through  the  clinic. 
They  may  also  have  applicability  to  broader  primary  health  care  settings. 

Summary  of  Learning 

Clients  do  not  always  utilize  services  they  have  access  to  and  clients  do 
not  always  have  convenient  access  to  services  they  need.  These  seem  to 
be  the  primary  issues  of  access  in  primary  health  care. 

1 . A central  theme  in  primary  health  care  is  community-based  care. 
Models  of  service  delivery  in  primary  health  care  environments  need  to 
include  a comprehensive  view  of  community  members’  health  needs. 
Within  a primary  health  care  context,  involvement  of  community 
members  is  expected.  This  includes  identification  of  “necessary” 
services  and  subsequent  planning  for  health  services  which  may 
require  negotiation  regarding  access  with  existing  service  providers 
and  health  system  decision-makers.  In  addition,  community  need  shifts 
with  the  changing  demographic  profile  of  community  members. 

2.  Access  to  primary  health  care  at  the  community-level  can  be  enhanced 
through  a variety  of  approaches.  Mobile  health  teams,  for  example, 
appear  to  work  in  rural  communities.  Clients  appear  to  be  satisfied  with 
the  service  provided  by  Nurse  Practitioners.  Access  to  medication 
management  information  through  collaborative  consultation  by  care 
providers  improves  medication  compliance.  Immunization  rates  can  be 
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improved  through  making  personal  (telephone)  contact  with  parents. 
At-risk  mothers  can  be  reached  through  establishing  personal 
relationships  with  them,  through  Home  Visitors. 

3.  Coordinated  and  integrated  delivery  of  programs  appears  to  improve 
access  to  service  by  clients — particularly  mental  health  services  and 
palliative  care  services.  For  example,  community-based  mental  health 
strategies  appear  to  successfully  improve  physicians’  capacity  to 
diagnose  and  treat  mental  health  problems,  reduce  time  of  diagnosis 
of  children’s  mental  health  disorders,  and  provide  opportunities  for 
therapy  for  newcomers  to  Canada.  It  appears  that  access  to  most 
primary  health  care  services  could  benefit  clients  through  increased 
coordination  and  integration  of  services.  However,  specific  strategies 
need  to  be  considered  on  an  individual  basis  since  coordination  and 
integration  strategies  may  require  significant  changes  to  the  ways  in 
which  services  are  currently  delivered.  Changes  in  practice,  while 
intended  to  improve  access,  could  create  temporary  barriers  to 
services  while  change  management  strategies  are  deployed. 

4.  Access  can  be  improved  through  publicity.  Providing  reminder  notices 
for  appointments  (e.g.,  immunization,  mental  health  appointments) 
improves  clients’  utilization  of  service.  It  is  reasonable  to  assume  that 
similar  promotion  strategies  could  be  applied  to  other  health  service 
areas.  Promotional  campaigns  have  the  potential  to  increase  the 
profile  of  programs.  With  increased  public  awareness  of  programs  and 
services,  perceptions  of  access  are  likely  to  improve.  Furthermore, 
appropriate  use  of  services  needs  to  be  advertised.  People’s 
perceptions  of  need  for  certain  services  can  be  influenced  by  anxiety 
concerning  their  own  health  or  the  health  of  their  loved  ones.  For 
example,  people  appear  to  access  services  at  the  Emergency 
Department  out  of  worry  and  perception  of  urgent  need  for  care  rather 
than  out  of  convenience. 

5.  The  current  fee  for  service  structure  discourages  collaboration  among 
health  professionals  (e.g.,  physicians/pharmacists, 
psychiatrists/pediatricians)  and  between  physicians  and  patients). 


rA 
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Integration 


Integration  and  coordination  are  regarded  as  essential  elements  of  a 
primary  health  system.  Integration  and  coordination  typically  involve  the 
cooperation  of  inter-disciplinary  and  inter-sectoral  teams  in  delivering 
coordinated  services.7  Integration  and  coordination  are  especially 
important  in  primary  health  care  which  considers  health  broadly.  Health 
services,  for  example,  encompass  the  continuum  of  physical,  mental,8  and 
spiritual  health  needs,  including  services  supporting  well-being  and  quality 
of  life  through  prevention  and  promotion  activities.  Research  suggests  that 
integration  has  the  potential  to  improve  cost-effectiveness  of  health 
services,  efficiency  and  productivity,  health  status,  and  patient/client 
satisfaction. 

Several  of  the  Alberta  Primary  Health  Care  projects  provide  these  types  of 
evidence  and  examples  of  both  vertical  integration  (along  the  continuum  of 
care)  and  horizontal  integration  (coordination  across  specific  health  care 
domains). 

Importance  and  Benefits  of  Information  Sharing 

One  of  the  most  influential  facilitators  of  an  integrated  health  system  is  the 
sharing  of  client  records/information.  Very  few  projects  developed 
strategies  to  achieve  a comprehensive  system  of  information  sharing. 
However,  many  projects  identified  that  without  the  capacity  to  share 
information  about  patients/clients  across  sectors  and  providers,  integration 
was  impeded  (e.g.,  8th  and  8th). 

Without  a common  client  information  system  it  is  difficult  to  address 
service  impact  or  program  effectiveness,  particularly  health  status  and 
population  health.  Lack  of  designated  boundaries  for  catchment  areas,  as 
the  8th  and  8th  Health  Centre  project  demonstrated,  also  exacerbates 
collection  of  information.  Another  project  demonstrated  that  sharing  of 
administrative  information  between  the  region  and  projects  can  also  be  a 
problem.  Integration  of  the  Misericordia  Health  Lifestyle  Education 
program  within  the  broader  political  and  regional  health  authority  delivery 
system  posed  challenges  in  that  budget  tracking  was  extremely  difficult 
due  to  several  layers  of  administration  and  requirements  of  regional 
funding  agreements.  Similar  difficulties  were  experienced  by  the 
Keeweetinok  Immunization  project  in  reconciling  project  records  with 
larger  system  records  and  differences  between  reporting  periods.  While 
individual  projects  can  be  nimble  and  track  day-to-day  expenses,  larger 


7 Provincial  Health  Council  of  Alberta  (1997). 

8 Mental  health  may  include  spiritual  health  for  some  populations. 
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financial  systems  are  more  geared  to  monthly  or  quarterly  reporting 
schedules. 

Recognizing  the  importance  of  the  creation  of  an  information  system  that 
supports  integration,  the  Northeast  Community  Health  Centre  introduced  a 
common  (paper-based)  client  record  in  all  service  areas  except 
Community  Health.  A common  client  record,  whether  chart  based  or 
electronic,  enables  all  health  professionals  involved  with  a particular  client 
access  to  the  same  health  related  information  about  that  client.  The 
Northeast  is  concerned  that  while  an  electronic  record  will  improve 
integration,  its  immediate  introduction  will  actually  impede  integration.  Like 
other  projects,  the  Northeast  discovered  that  some  existing  electronic 
database  systems  were  not  compatible  with  one  another.  Therefore, 
access  to  the  record  can  be  cumbersome  and  time  consuming. 


During  this  time  period  she  is  listed  as  having  $360.00  in  overtime  despite  her 
negative  income  for  regular  hours  worked... this  is  due  to  system  end-of-year 
calculations  that  are  built  into  their  tracking  system  that  do  not  necessarily 
correspond  to  actual  salaries  (Ausford,  2000). 


At  Sunridge,  analysis  of  the  information  gathered  over  time  will  aid  in  the 
integration  of  external  services  with  those  provided  by  UCMC-Sunridge. 
The  information  system  selected  for  use  in  the  evaluation  of  clinics  has  the 
ability  to  integrate  information  about  primary  and  secondary  services, 
Alberta  Health  and  Wellness  billing  data,  and  laboratory  reports.  In 
addition,  the  Sunridge  project  has  developed  a manual  to  assist  in 
evaluating  information  systems. 

With  respect  to  the  value  of  information  sharing  among  health 
professionals,  family  physicians  in  the  Shared  Mental  Health  project  felt 
more  enabled  regarding  the  management  of  mental  health  in  a primary 
care  setting.  They  appreciated  direct  assistance  with  assessment, 
diagnosis,  and  treatment  planning.  Access  to  psychiatrists  for  medication 
advice  and  consultation  was  also  highly  valued  by  physicians.  In  this 
project,  family  physicians  believed  they  made  fewer  overall  and  more 
appropriate  referrals  to  outside  agencies  as  a result  of  having  access  to  a 
mental  health  professional.  They  appreciated  having  increased  access  to 
a mental  health  professional  through  a)  direct  referral,  b)  joint  sessions 
with  a mental  health  professional  and  a patient,  c)  consultation  with  a 
mental  health  professional,  and  d)  rapid  referral  to  appropriate  community 
resources.  Increased  access  resulted  in  physicians  feeling  that  they  had 
increased  skill  and  confidence  in  dealing  with  mental  health  issues.  The 
patient  appreciated  receiving  mental  health  therapy  in  the  familiarity  of  the 
physician’s  office.  However,  mental  health  clinicians  commented  on 
difficulties  serving  between  4-6  physicians’  offices,  so  the  long-term 
advantages  and  disadvantages  of  such  a delivery  model  need  to  be 
addressed.  Providing  mental  health  resources  through  group  or  clinic 
practice  may  work  better  than  through  a solo  practitioner. 
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Overall,  physicians  improved  in  their  management  of  mental  health  issues 
through  differentiated  understanding  of  symptoms  and  problems.  Physicians 
shifted  from  being  solely  “prescribers”  and  “fixers”  to  being  enablers  and 
facilitators  (Bakal,  2000). 

Increased  access  to  diagnostic  and  treatment  information  from  oncologists 
by  family  physicians  and  other  caregivers  (e.g.,  home  care  coordinators) 
facilitated  the  transfer  of  palliative  patients  across  levels  of  care  in  the 
Calgary  Palliative  Care  project.  The  fact  that  other  physicians  at  the  Tom 
Baker  Cancer  Clinic  requested  to  use  the  communication  template 
produced  by  the  project  attests  to  its  applicability  beyond  palliative  cancer 
patients.  It  also  emphasizes  that  expedient  information  sharing  is  an 
important  aspect  of  being  able  to  provide  quality  care  to  palliative  patients. 
Part  of  the  patients’  perception  of  quality  care  was  the  level  of  confidence 
they  had  about  their  family  physician  having  timely  access  to  treatment 
information. 

Strategies  for  Information  Sharing 


Software  Adaptation 

The  Collectives  project  demonstrated  development  of  improved  ways  of 
gathering  data,  sharing  information,  and/or  communicating  information 
through  the  system.  Adaptation  of  computer  software  enabled  community 
pharmacists  to  document  and  share  information  on  medication  reviews, 
health  issues  identification,  monitoring,  and  follow  up  on  issues  in  the 
community.  This  database  can  be  used  with  other  collaborative  practice 
teams  and  with  other  parts  of  the  health  system  with  proper  safeguards  for 
patient  confidentiality.  The  database  would  also  be  useful  for  program 
monitoring  and  service  planning  within  health  regions. 

Some  of  the  patients  involved  in  the  Collectives  gave  consent  to  have  their 
medication  review  information  shared  among  pharmacists  within  the 
Collectives  and  outside  the  Collectives.  The  project  then  developed 
mechanisms  to  integrate  publicly-funded  and  non-publicly  funded  services 
through  obtaining  existing  medications  from  community  pharmacists 
outside  the  Collectives,  communicating  with  them  as  needed,  and  advising 
them  of  important  medication  issues.  Procedures  were  developed  for  the 
transfer  of  medication  lists  from  acute  care  to  the  Collectives.  These 
developments  facilitated  information  sharing  and,  ultimately, 
coordinated/integrated  service  provision  among  family  physicians, 
pharmacists,  and  the  home  care  case  manager. 

The  Communication  Passport/Communication  Template 


The  Lakeland  Palliative  Care  pilot  demonstrated  successful  strategies  to 
achieve  communication  among  providers,  for  example,  through  the 
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Communication  Passport  which  facilitated  the  smooth  transition  from  one 
care  provider  to  another  (e.g.,  shift  from  acute  care  to  palliative  care  at 
home).  The  Communication  Passport  allowed  the  patient  to  feel  in  control 
of  his  or  her  care.  It  also  allowed  the  care  provider  to  know  what  others 
had  suggested  with  respect  to  care  plans.  As  well,  communication  across 
the  multi-disciplinary  team  was  achieved  through  adaptation  of  the 
recording  system  typically  used  in  the  Health  Centre  in  Bonnyville  (i.e.,  in 
relation  to  pain  and  symptom  management).  The  project  learned  that 
communication  protocols  need  to  be  negotiated  with  nurses  and  that 
burdens  of  documentation  must  be  kept  minimal  for  nursing  staff. 

The  standardized  communication  template  used  in  the  Calgary  Palliative 
project  serves  as  a second  example  of  a successful  strategy  for 
information  sharing. 

Telehealth 

Some  services  offered  or  enhanced  through  telehealth  included 
educational  inservices,  speech  therapy,  prenatal  education,  dental  health 
for  children,  occupational  health  orientation,  human  resource  orientation, 
intervention  and  assessment  strategies,  and  communication  abilities  for 
community  health  nurses.  Insufficient  data  make  it  difficult  to  conclude 
whether  or  not  telehealth  technology  will  be  instrumental  in  facilitating 
planning  by  health  stakeholders  and  the  public.  From  the  few  educational 
sessions  that  took  place  with  stakeholders  during  the  project,  it  appears 
that  telehealth  has  the  potential  to  facilitate  collaborative  processes. 


Telehealth  technology  in  Trout  Lake,  Peerless  Lake  and  Red  Earth  Creek  has 
presented  an  opportunity  for  KLRHA  [Keeweetinok  Lakes  Regional  Health 
Authority]  to  provide  people  in  these  communities  access  to  a more  diverse  health 
system  than  they  ever  dreamed  possible  (Wynne,  2000). 


Consultation  Sessions 


Further  examples  that  illustrate  the  effect  of  increased  access  to 
information  by  providers  and  improved  access  to  health  service  by  clients 
included  the  C.O.P.E.  project  in  which  providers  held  consultation 
sessions  that  involved  the  contributions  of  school-based  professionals, 
Learning  Specialists,  Family  Liaison,  and  family  members.  A broader 
range  of  contextual  information  about  the  child  facilitated  a more 
comprehensive  diagnosis  of  children’s  mental  health  disorders.  (This  point 
is  elaborated  further  in  the  next  section.) 

Health  professionals  involved  in  the  Collectives  project  (i.e.,  family 
physician,  community  pharmacist,  and  home  care  nurse)  also  gained 
access  to  information  about  medication  use  of  patients  through  their 
collaborative  consultations  and  ongoing  communication. 
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Clinical  Practice  Guidelines  Development 

The  Strengthening  Multidisciplinary  Teams  project  formed  multidisciplinary 
working  groups  to  consider  best  evidence  about  best  practice,  identify 
chronic  disease  management  problems,  set  priorities,  prepare 
multidisciplinary  care  plans,  and  solicit  input  from  experts.  Team 
functioning  was  monitored  and  effects  on  process  and  selected  outcomes 
were  evaluated. 

The  creation  of  a virtual  team  community  for  each  working  group  was 
facilitated  through  internet-based  information  technology  to  link  all 
members  to  a project  website.  This  project  demonstrated  that  electronic 
decision  support  tools  and  online  access  to  information  can  support 
multidisciplinary  involvement  and  implementation  of  practice-based  care 
plans. 

Figure  2.  Screen  From  Health  Care  Teams  Site 
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While  these  examples  focus  on  access  to  information  by  providers, 
increase  in  information  had  a direct  impact  on  improved  access  to  services 
by  patients/clients.  These  examples  also  illustrate  the  close  relation 
between  access,  integration,  and  quality  of  health  service. 

Movement  Toward  Integrated  Systems 


Several  projects  emphasized  that  integration  is  achieved  only  with  the 
support  of  appropriate  structures  and  processes.  Some  projects  were  able 
to  demonstrate  integration  successfully  because  of  either  having 
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appropriate  structures/process  in  place,  or  creating  them  as  part  of  their 
implementation  process. 

Leadership 

The  Leadership  Team  in  the  Calgary  Palliative  Care  project  learned  that 
structures  and  processes  need  to  be  in  place  to  support  integrated  service 
delivery.  Family  physicians  must  be  connected  to  oncologists  in  timely  and 
effective  ways.  Sharing  of  electronic  health  records  may  effectively  and 
efficiently  supplement  and/or  replace  print-based  information.  (Issues  of 
privacy,  of  course,  will  need  to  be  addressed.)  The  Leadership  Team, 
itself,  was  a successful  example  of  multiple  provider  groups  working 
across  disciplines  to  improve  palliative  care.  The  Team  demonstrated  that 
a common  vision,  purpose,  and  commitment  go  a long  way  to  change  how 
services  are  provided.  Requests  from  other  providers  to  examine  and 
explore  the  model  of  integrated  service  delivery  employed  in  this  project 
confirmed  to  the  Leadership  Team  that  they  were  on  the  right  track. 

With  respect  to  integration  within  other  palliative  care  programs,  the 
Consultation  Team  in  the  Lakeland  Palliative  Care  project  is  networked  to 
provincial  and  national-level  associations  dedicated  to  advancing  palliative 
care.  These  linkages  keep  the  Consult  Team  informed  of  best  practices 
and  allow  them  to  share  learning  with  other  team  members  and 
incorporate  advancements  into  the  regional  palliative  care  program.  In  this 
manner,  the  Consultation  Team  can  play  a leadership  role  in  palliative 
care  in  the  region. 

Leadership  proved  very  important  in  the  Strengthening  Multidisciplinary 
Teams  project.  In  an  environment  where  practitioners  were  limited  in  the 
time  they  could  give  to  the  project,  possibly  for  reimbursement,  and  other 
workload-related  reasons,  it  proved  difficult  to  find  participants  that  were 
willing  to  motivate  the  team. 

Integrated  Service  Delivery 

The  Northeast  Community  Health  Centre  project  demonstrated  integrated 
service  delivery  with  other  parts  of  the  health  system  and  with  other  key 
stakeholders.  Formal  agreements  were  established  with  five  external 
agencies  and  organizations  to  collaboratively  deliver  services.  Four  of 
these  agreements  involved  the  creation  of  service  provider  positions  at  the 
NECHC  initially  set  to  end  May  31 , 2000.  Most  of  these  agreements  are 
now  either  continuing  or  are  being  negotiated  for  continuance.  The  fifth 
agreement  involved  the  creation  of  an  Asthma  Centre  and  other  initiatives 
for  a longer-term  period. 


Significant  learning  about  integration  was  gained  from  these  efforts — that 
integration  be  considered  within  the  broader  social/political  context,  and  that  it 
involve  providers  and  the  larger  regional  system  within  which  services  are  to  be 
delivered  (Wilson  et  al,  2000). 
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The  8th  and  8th  Health  Centre  also  demonstrated  integrated  service 
delivery.  Integration  was  facilitated  by  the  development  of  personal 
relationships  among  staff  members  within  the  Centre. 

In  the  CUPS  project,  the  integration  of  the  NP  and  the  implementation  of 
the  collaborative  practice  model  was  the  foundation  for  all  other  project 
successes.  Integration  within  CUPS  and  between  CUPS  and  health 
system  was  achieved  through  the  NP  by  involvement  in  the  interagency 
groups.  CUPS  staff  meet  regularly  with  other  inner  city  providers  to  avoid 
duplicates  and  promote  continuity  of  care.  (Specific  results  indicate  that 
the  implementation  of  the  NP  role  has  not  greatly  increased  the 
collaboration  within  CUPS  due  to  limited  clinical  time  available). 
Coordination  between  CUPS  and  the  local  health  systems  occurred  at 
several  levels.  Blurred  roles  between  the  NP  and  other  nurses  resulted.  In 
addition,  the  NP  joined  other  health  organizations,  detox  centres,  and 
police  services  from  the  Downtown  Interagency  Group. 

Determinants  of  Health 


The  four  major  partners  of  the  Alexandra  CHC  reported  that  partnerships 
were  formed  with  the  CHC  because  of  its  community  outreach  initiatives 
and  potential  for  optimizing  both  individual  and  community  health.  Based 
on  the  results  of  the  internal  evaluation  of  the  Alexandra  CHC,  the 
independent  evaluators  reported  that  it  appears  that  its  service  delivery 
model  allows  for  integration  and  coordination,  both  with  other  parts  of  the 
health  system  and  with  other  stakeholders.  Through  the  use  of  a 
multidisciplinary  approach,  there  appears  to  be  the  ability  to  integrate  and 
coordinate  services  within  the  Alexandra  CHC  and  beyond.  The  model  of 
service  delivery  is  also  appropriate  for  building  linkages  with  the 
community.  The  Centre  takes  a broad  view  of  health  and  places  particular 
emphasis  on  the  physical,  as  well  as  social  and  psychological  well-being 
of  clients. 


They  are  more  resourceful.  It’s  not  a busy  doctor’s  office  where  they  are  willing  to 
just  give  you  drugs  and  send  you  out  the  door.  Instead  they  suggest  chiropractor, 
massage  or  psychiatric  help.  Most  doctors  have  a tendency  to  look  after  their 
patients  without  suggesting  other  options  (Conte  et  al,  Project  Report  2000). 


Like  Alexandra,  the  Healthy  Okotoks  Coalition  (HOC)  recognized  and 
addressed  the  determinants  of  health,  including  social,  cultural,  and 
economic  factors  that  affect  the  health  status  of  individuals  and  the 
community  of  Okotoks  overall. 
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Community  Consultation 

In  the  East  Central  project  strategic  direction  for  the  organization  and 
delivery  of  primary  health  services  was  set  by  the  Board.  Links  were  made 
to  the  regional  Business  Plan  and  to  provincial  goals  identified  by  Alberta 
Health  and  Wellness.  Success  indicators  for  the  development  of  the 
integrated  primary  health  system  were  identified  by  the  region.  Existing 
community  and  facility  managers  facilitated  linkages  between  the  region 
and  the  communities.  Functional  integration  of  the  project  into  ongoing 
community  health  activities  and  integration  between  the  communities  and 
the  region  occurred  on  an  ongoing  basis.  Regional  staff  in  East  Central 
and  community  stakeholders  identified  two  gaps  to  integrated  services: 
lack  of  integration  of  physicians  into  the  project  and  disconnect  between 
service  delivery  based  on  the  medical  model  into  a primary  health  services 
framework.  The  project  emphasized  the  need  to  integrate  community 
health  planning  and  service  responses  with  the  surrounding  areas  of  the 
four  pilot  communities. 

Intersectoral  Collaboration 


The  Healthy  Okotoks  Coalition  demonstrated  a great  degree  of 
intersectoral  collaboration  among  the  Town  of  Okotoks,  key  stakeholder 
groups  and  organizations,  and  the  Headwaters  Health  Authority.  The 
Healthy  Okotoks  Coalition  itself  collaborated  with  an  environmental  lobby 
to  encourage  responsible  pesticide/herbicide  use  in  the  community,  and  it 
has  linked  with  local  expertise  to  increase  the  awareness  of  Fetal  Alcohol 
Syndrome  (FAS). 

The  C.O.P.E.  program  demonstrated  successful  cooperation  and 
collaboration  between  among  school-based  professionals  (the  education 
system)  and  medical  professionals  (the  health  system).  C.O.P.E. 
demonstrated  integration  of  medical  services  into  school  communities. 

The  service  delivery  model  implemented  in  the  Lakeland  Palliative  pilot 
involved  the  cooperation  of  physicians,  nurses  (acute  care  and  home 
care),  home  support  aides,  social  workers,  occupational  and  respiratory 
therapists,  community  volunteers,  and  others  (e.g.,  funeral  directors). 

The  Healthy  Families  Program  (Capital)  demonstrated  integration  of  its 
program  (or  components  of  it)  into  existing  services  provided  by 
community-based  agencies.  Building  upon  existing  delivery  systems  was 
believed  to  increase  capacity  within  the  existing  community  agency 
system.  The  collaborative  partnership  approach  has  helped  to  ensure  an 
integrated  support  system  for  families  enrolled  in  the  Program.  The 
involvement  of  multi-disciplinary  representatives  on  the  committees  helps 
support  the  delivery  of  integrated  services  to  families. 
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“I  am  somewhat  isolated  as  a single  parent  and  it  is  awesome  that  a Home  Visitor 
offers  socializing  and  interaction  opportunities  within  the  community.  My  worker  is 
superb  and  finds  out  things  for  me  that  otherwise  I wouldn’t  know  about”— Healthy 
Families  parent  (MacDonald,  2000). 


Interdisciplinary  Teams 

In  the  Shared  Mental  Health  project,  the  project  was  successful  in 
improving  links  between  primary  care  and  mental  health  care.  Physicians, 
mental  health  clinicians,  and  psychiatrists  worked  together  collaboratively 
in  a primary  care  setting.  The  physician  remained  the  first  point  of  contact 
for  the  client.  The  independent  evaluators  noted  that  it  would  be 
advantageous  for  future  integration  to  formalize  the  mix  of  professionals 
through  training  programs  and  include  opportunities  for  both  medical  and 
non-medical  health  professionals  to  work  together  (e.g.,  family  practice 
clinic  with  opportunities  for  residents  in  psychiatry,  nursing,  psychology, 
social  work,  and  other  mental  health  specialties).  More  work  needs  to  be 
done  to  improve  integration  with  secondary  care.  Changes  required 
include  becoming  more  interdisciplinary  (blurring  of  professional 
boundaries)  versus  multidisciplinary  (involving  different  professionals  who 
maintain  their  professional  boundaries). 

In  Health  Authority  5 informally  structured  primary  health  care  teams 
formed  to  deliver  services  to  four  rural  communities.  Strong  and  positive 
relationships  were  reported  among  team  members  and  between  staff  and 
community  members.  Teams  linked  with  other  professionals  including 
laboratory  and  pharmacy  services,  health  promotion,  and  prevention. 

In  the  Collectives  project,  better  use  of  existing  resources  in  assessment 
and  treatment  of  patients  with  complex  health  needs  was  demonstrated  in 
that  physicians  were  used  to  diagnose  problems.  While  monitoring  and 
follow  up  was  done  increasingly  by  non-physician  members  of  the 
Collectives  (i.e.,  pharmacists  and  home  care  case  managers).  This 
strategy  indicated  a more  efficient  use  of  existing  resources. 

Interdisciplinary  collaboration  occurred  at  the  program  planning  and 
delivery  level  in  the  Misericordia  Health  Lifestyle  Education  program,  and 
as  well  where  several  professional  groups  were  involved  in  the  project, 
particularly  the  project  director  (family  physician),  teaching  staff  (dietician 
and  exercise  psychologist),  and  the  Certified  Fitness  Consultants. 

Strengthening  Multidisciplinary  Teams  project  working  groups  were  able  to 
recognize  high  quality  distillates  and  guidelines  summarizing  best 
evidence  and  were  comfortable  accepting  these  documents  as  a starting 
point.  Team  care  plans  were  developed  for  diabetes  and  osteoarthritis. 
Translating  evidence-based  practice  into  practice-based  evidence  proved 
more  difficult.  Front-line  practitioners  felt  the  guidelines  were  somewhat 
“vague.”  They  felt  they  required  more  explicit  guidelines  for  evidence  to  be 
useful  in  practice. 
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Co-Location  of  Services 


While  integrated  service  delivery  had  not  occurred  as  planned,  the  Airdrie 
Regional  Health  Centre  (ARHC)  demonstrated  that  collaboration/co- 
location of  services  facilitated  increased  efficient  service  delivery, 
improved  communication,  cooperation,  reduced  duplication  of  effort,  and 
simplified  referral  processes  among  programs  housed  at  the  Centre.  Co- 
location  was  also  perceived  to  result  in  increased  convenience  for  clients 
because  of  increased  visibility,  and  increased  walk-in  traffic  particularly  for 
Family  Services.  A broader  clientele  was  attracted  (e.g.,  Hutterites  were 
more  likely  to  visit  the  Centre).  The  physical  structure  of  the  building  itself 
facilitated  increased  collaboration  between  Centre  staff  and  community 
and  outreach  workers. 

The  WestView  Healthy  Families  program  was  co-located  with  the  existing 
public  health  nursing  program.  This  was  seen  as  providing  continuity 
between  the  program  and  other  Well  Baby  programs  offered  by  the  Health 
Unit.  An  added  benefit  was  the  ongoing  support  the  Public  Health  nurses 
provided  to  Family  Visitors,  especially  in  relation  to  healthy  growth  and 
development  of  the  children. 

And  finally,  the  Northeast  Community  Health  Centre,  a major  project  in 
Capital  region,  demonstrated  co-location  of  Emergency  services  with  a 
host  of  other  services  provided  at  the  Centre. 

Involvement  of  Stakeholders 


Several  projects  involved  stakeholders  in  the  formulation  of  the  project 
itself,  in  its  implementation,  and  certainly  in  contributing  to  a determination 
of  its  success.  Some  projects  involved  stakeholders  in  an  integral  way — 
that  is,  in  actually  planning  the  services  that  would  be  delivered  to  them. 
For  example,  in  the  Collectives  project  15%  of  care  issues  were  identified 
by  patients. 

Facilitators  of  Integration 

A variety  of  projects  highlighted  several  facilitators  of  integration. 

Cross-disciplinary  Understanding  and  Training 

In  the  Calgary  Palliative  project,  three  initiatives  were  undertaken  to 
support  integrated  delivery  of  palliative  care  services  in  CRHA.  Knowledge 
was  gained  about  education,  training,  and  communication  needs  of  family 
physicians,  oncologists,  community  care  coordinators,  patients  and 
families  as  a first  step  to  learning  what  providers  need  to  know  about 
expectations  of  other  providers  including  patients/families.  This  information 
provides  a useful  first  step  to  understanding  what  it  might  take  to  work 
successfully  together.  The  needs  assessment  process,  therefore,  proved 
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to  be  a productive  technique  to  begin  moving  toward  successful 
integration  of  palliative  care  services. 

C.OP.E.  demonstrated  that  it  can  serve  as  an  ideal  training  ground  for 
both  medical  practitioners  and  school-based  professionals. 

The  group  work  involved  in  drafting  the  Clinical  Guidelines  for  ten 
conditions  at  CUPS  served  as  an  excellent  team  building  activity  for  the 
professional  staff  and  Nurse  Practitioner  at  CUPS.  It  also  encouraged 
discussion  about  role  clarification  and  contribution  to  an  integrated  and 
collaborative  model  of  service  delivery — the  basis  for  a continuous  quality 
improvement  process. 

Cross-training  of  staff  also  facilitates  integration.  For  example,  the  skills  of 
all  reception  and  clerical  staff  at  the  8th  and  8th  Health  Centre  were 
augmented  to  address  narrowly  defined  job  functions  based  on  program 
area  and  to  create  an  administrative  support  team. 

Similarly,  the  Lakeland  Palliative  project  demonstrated  that  training  and 
education  provided  to  care  providers  and  community  volunteers  is  key  to 
building  capacity  for  providing  services  in  an  integrated  way.  It  requires 
that  providers  know  each  others’  scopes  of  practice.  (This  information  has 
been  identified  in  the  Policies  and  Procedures  Manual  prepared  by  the 
project.) 

Leadership 

The  Northeast  Community  Health  Centre  project  demonstrated  the 
importance  and  influence  of  leadership  on  integration.  For  example,  Board 
commitment  was  critical  for  establishing  operations  of  the  NECHC  within 
requirements  of  overall  health  system  and  regional  structure.  Project  team 
members  reported  that  the  process  of  developing  relationships  across 
service  areas  and  among  individual  staff  has  been  evolutionary.  Centre- 
wide initiatives  such  as  staff  leadership  meetings  have  been  introduced  to 
build  integration  among  staff  and  to  improve  integrated  delivery  of 
services. 

Other  facilitators  of  integration  that  have  been  identified  by  various 
projects  include 

• congruence  between  existing  reporting/organizational  structures 
and  program  mandates/geographical  catchment  areas; 

• physician  reimbursement  mechanisms  with  incentives  for 
consultation; 

• a clear  definition  and  regional  focus  on  primary  health  care; 

• funding  for  the  coordination  of  multi-disciplinary  service  delivery, 
particularly  in  rural  Alberta,  and  in  areas  where  need  for  integrated 
service  is  required  (e.g.,  increased  need  for  palliative  care); 

• interdisciplinary  representatives  on  committees; 
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• integrated  service  delivery  models  (show  Lakeland  Palliative, 
C.O.P.E.,  Calgary  Palliative)  that  communicate  to  the  staff  and  the 
public  how  services  will  be  organized  and  delivered; 

• community  awareness  of  the  range  of  services  offered; 

• involvement  and  commitment  of  physicians;  and 

• services  that  reflect  the  determinants  of  health. 

Indicators  of  Integration 

In  the  Sundridge  project,  twenty  indicators  of  integration  (from 
patient/provider  perspective)  were  identified  and  five  were  selected  for 
inclusion  in  the  final  CQI  evaluation  template.  They  include 

• patients’  knowledge  of  what  needs  to  be  done  after  consultation, 

• amount  of  patient  information  received  from  other  providers, 

• summaries  of  patient  records  provided  to  other  physicians, 

• knowledge  of  patient  care  received  elsewhere,  and 

• physicians’  knowledge  of  specialists. 

Overcoming  Barriers  to  Integration 


Turf  Protection/Loss  of  Control 


Projects  identified  several  barriers  to  successful  integration  including 

• organizational  management  structures  that  do  not  encourage 
integrated  service  delivery  (i.e.,  different  reporting  structures, 
budgets,  geographical  catchment  areas), 

• rear  of  losing  control  of  program  areas  to  local  decision-making 
bodies, 

• lack  of  coterminous  catchment  areas  for  programs  offered,  and 

• lack  of  awareness  of  physicians  and  consumers  of  the  range  of 
services  available  (e.g.,  Diabetes  Education  Centres  within  Capital 
region). 

However,  one  of  the  primary  challenges  to  successful  integration 
expressed  by  projects  is  that  of  turf  protection — most  commonly  among 
key  providers.  Successful  strategies  were  demonstrated  in  overcoming  turf 
barriers. 

The  approach  used  by  the  Consultation  Team  in  the  Lakeland  Palliative 
project  was  an  exemplar  approach  to  encourage  family  physicians  to  draw 
on  the  expertise  and  support  of  the  Palliative  Care  Physician  Consultant 
and  Palliative  Care  Nurse  Consultant.  The  Physician  and  Nurse 
Consultant  essentially  assumed  a stance  of  “humble  expert,”  suggesting 
and  offering  advice,  but  careful  to  allow  the  family  physician  to  feel  and  be 
in  control  of  the  patient  care.  This  approach  facilitated  break  down  of  turf 
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barriers — physician  to  physician  and  physician  to  nurse.  The  personalities 
of  the  Consultation  Team  were  a key  factor  in  successful  integration  of 
recommendations  and  advice  by  the  Consultation  Team  into  physicians’ 
and  nurses’  practices.  Signs  that  this  approach  had  worked  successfully 
were  indicated  by 

• physicians  alerting  the  Consultation  Team  to  anticipated  referrals; 

• requests  for  consultation  from  physicians  outside  the  pilot  and  also 
from  outside  the  region; 

• rural  physicians’  acknowledgement  that  the  Nurse  Consultant  was 
an  important  part  of  the  team; 

• opportunity  to  apply  new  knowledge  and  demonstrate  positive 
impacts  of  the  pilot  on  patients  to  other  staff;  and 

• consideration  by  physicians  and  other  care  givers  (e.g.,  nurses)  of 
alternatives  to  conventional  palliative  care  treatment. 

Further  indication  of  success  are  the  linkages  made  between  the  project  to 
palliative  care  groups  throughout  the  region,  with  other  regions,  and  with 
the  Alberta  Cancer  Board.  The  Project  Coordinator  was  invited  to  join  the 
Palliative  Care  Association  of  Alberta;  the  Physician  Consultant  the 
Alberta  Cancer  Board  Advisory  Council.  Several  requests  for  information 
have  originated  across  the  province,  and  one  from  the  U.S. 

A second  very  successful  example  of  overcoming  turf  barriers  was  the 
Strengthening  Multidisciplinary  Teams  project.  This  project  demonstrated 
the  importance  of  involving  front-line  practitioners  in  transforming  best 
evidence  into  best  practice.  While  the  project  indicated  that  it  is  very 
difficult  to  draw  on  the  time  of  front-line  staff,  it  is  critical  in  the  application 
of  best  practice  guidelines  to  practice. 


Care  plan  development  is  not  a trivial  process  and  requires  considerable  group 
interaction  to  achieve  a mutually  satisfactory  result.  Working  group  members 
require  protected  time,  and  institutional  support  (Hayward,  2000). 


The  project  emphasized  that  health  professionals  require  ongoing  support 
to  integrate  technology,  guidelines,  or  any  other  changes  in  practice.  They 
must  be  reimbursed  for  their  time  and  released  from  regular  duties  so  that 
participation  is  not  viewed  as  an  overload  assignment. 

Publicly  and  Non-publicly  Funded  Services 

The  Collectives  project  developed  mechanisms  to  integrate  publicly- 
funded  and  non-publicly  funded  services  and  procedures  to  transfer 
medication  lists  from  acute  care  to  the  Collectives.  (Note:  community 
pharmacists  in  Alberta  are  paid  through  individual  drug  dispensing  fees  set 
by  Alberta  Health  and  Wellness  and  are  based  on  the  cost  of  the  drug,  on 
a fee-for-service  basis.  They  are  not  paid  directly  for  what  is  known  as 
“cognitive  services”  such  as  consultation  with  patients/clients  or  with  other 
health  professionals.)  Lists  of  existing  medications  were  obtained  from 
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community  pharmacists  outside  the  Collectives.  Communication  between 
Collectives  members  and  community  pharmacists  occurred  as  needed. 

As  mentioned  previously,  the  C.O.P.E.  project  highlighted  that  differential 
billing  rates  and  opportunities  to  bill  create  a disincentive  for  integrated 
services  at  this  time  in  Alberta  (i.e.,  psychiatrists  can  be  paid  to  consult 
with  the  physician  and/or  psychologist  in  the  absence  of  the  child  while  the 
family  physician  cannot). 

Summary  of  Findings 


1.  Information  gathering  and  information  sharing  are  critical  to  integration 
of  health  services  in  a primary  health  care  setting.  Without  information 
it  is  difficult  to  coordinate  services  and  assess  program  effectiveness 
or  impact  on  clients.  Compatible  and  inter-operable  information 
systems  and  other  tools  that  facilitate  both  vertical  and  horizontal 
integration  need  to  be  developed. 

2.  Perception  of  quality  of  care  by  clients  can  be  influenced  through 
integration  of  services.  Coordinated/integrated  services  increase 
patients’  or  patients’  family’s  confidence  that  their  health  service 
provider  (often  the  family  doctor)  has  access  to  important  information 
that  can  inform  treatment.  The  patient  senses  that  the  system  can 
provide  continuous  care. 

3.  Integrated  service  delivery  is  supported  by  leadership  appropriate 
structures  and  processes.  A number  of  projects  demonstrated  that 
integration  forces  change.  Therefore,  change  management  strategies 
need  to  be  instituted  in  conjunction  with  major  integration  initiatives. 

4.  Integration  is  often  attempted  through  the  formation  of  interdisciplinary 
(blurring  of  professional  responsibilities)  and  multidisciplinary 
(maintenance  of  professional  boundaries)  teams  of  health  providers. 
Relationship  between  team  members  is  important  and  can  often  be 
facilitated  through  clear  definition  of  roles  and  responsibilities.  It  is 
important  teams  include  front-line  providers  so  that  planned  change 
becomes  change  in  practice. 

5.  Examples  of  successful  integration  appear  to  accommodate  the  need 
for  individual  practitioners  to  maintain  control  while  maintaining 
collegial  environments  where  professionals  are  willing  to  seek  advice 
across  professional  lines  of  responsibility.  In  addition,  establishing 
linkages  among  stakeholders  (patients,  health  professionals,  system 
administrators,  public,  etc.)  helps  develop  support  for  new  approaches 
to  integrated  services. 
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Quality 


Quality  is  an  important  aspect  of  primary  health  care.  While  stakeholder 
perception  is  a critical  factor  in  assessment  of  quality,  it  is  clear  that  quality 
is  a complex  interplay  between  a variety  of  factors:  client  satisfaction, 
providers’  definitions  of  technical  appropriateness  of  interventions,  and 
management  of  needs  and  resources.  Quality  also  appears  to  be  closely 
related  to  perceptions  of  accessibility  of  health  services. 

Several  themes  have  emerged  from  projects  that  shed  light  on  the 
dimension  of  quality  as  it  relates  to  primary  health  care. 

Client  Satisfaction 

Often,  determination  of  quality  is  viewed  from  the  perspective  of  the 
client.9  In  one  way  or  another,  all  independent  evaluators  assessed  the 
dimension  of  quality  according  to  this  indicator. 

Quality  and  Access 

There  appears  to  be  a close  relationship  between  quality  and  access.10  In 
the  Health  Authority  5 Rural  Health  project,  for  example,  users  of  regional 
services  consistently  rated  services  as  “very  good”  or  “excellent.” 

However,  statistical  analysis  of  data  collected  from  community  members 
after  the  introduction  of  mobile  health  teams  to  rural  communities  suggests 
that  community  members  perceived  the  quality  of  services  to  be  higher 
following  the  introduction  of  mobile  teams. 

Because  fewer  people  were  travelling  out  of  the  region  to  access  primary 
health  services,  indications  were  that  residents  in  the  rural  communities 
were  making  use  of  the  resources  provided  by  the  mobile  teams  (the 
primary  intervention  strategy  used  in  the  project).  Numbers  of  patients  per 
clinic  per  week  ranged  from  19  in  Cremona,  to  3.6  per  clinic  in 
Youngstown.  Young  children  and  seniors  were  the  largest  groups  utilizing 
services.  (Utilization  in  most  of  mobile  clinics  at  the  four  demonstration 
sites  had  increased  and  leveled  off  by  project  close.)  As  one  community 
member  stated,  “It  reminds  me  of  the  good  old  days  when 
Dr.  Anderson  lived  at  Wardlow.  Came  up  on  the  railway  speeder 
many  times.  I consider  this  to  be  a good  thing.  ” 


9 Client  here  includes  reference  to  patients  and  users  of  the  health  system  generally. 

10  Supported  in  the  literature  (Donabedian,  1998;  Haddad,  Fournier,  Machouf,  and  Yatara,  1998). 
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Positive  client  satisfaction  was  generally  reported  at  the  community  health 
centres  (CHCs)  involved  in  the  Alberta  Primary  Health  Care  project. 
According  to  the  independent  evaluator,  the  fact  that  a large  number  of 
clients  at  the  Alexandra  CHC  reported  that  they  did  not  seek  services 
outside  the  Centre  suggested  that  they  were  satisfied  with  services 
available  and  received  there. 


Other  places  are  more  impersonal,  more  business-like,  but  the  Alex  is  people 
centred.  There’s  something  different  here.  I think  it  should  be  a model  for  other 
clinics.  The  staff  are  absolutely  wonderful  and  you  get  the  feeling  that  they  like 
what  they  do  (Acton  et  at,  Project  Report,  2000). 


Similarly,  the  majority  of  clients  (93%)  at  the  8th  and  8th  Health  Centre  were 
very  satisfied  with  services.  Actions  that  could  be  taken  to  improve  quality 
as  recommended  by  clients  included  reducing  wait  times,  improving 
signage,  free  access  to  supplies  (e.g.,  medication)  and  equipment  for 
those  who  could  not  afford  them,  and  improved  flow  of  patients  through 
the  Centre.  In  some  fashion,  these  improvements  generally  speak  to 
improved  access  to  health  services. 

The  Northeast  CHC  also  reported  general  client  satisfaction  with  services, 
but  relying  on  client  satisfaction  data  created  some  difficulties  at  the 
Centre.  Because  the  Centre  targets  “at  risk”  populations,  access  to 
services  may  be  limited  for  those  presenting  who  are  not  considered  high 
risk  patients/clients. 

Clients’  perceptions  of  quality  of  service  may  also  have  been  influenced  by 
access  to  services  as  evident  in  the  Lakeland  Palliative  Care  project 
where  24-hour  access  to  consultation  advice  from  the  Physician 
Consultant  and  the  Nurse  Consultant  about  pain  management  was 
provided  to  patients  and  families  along  with  24-hour  care  by  home  care 
nurses. 

Input  from  clients  at  the  Elnora  Health  Centre  (into  which  a Nurse 
Practitioner  was  incorporated)  indicated  increased  satisfaction  with 
services.  Independent  evaluation  suggested  that  satisfaction  increases 
may  be  attributed  to  the  fact  that  the  community  had  previously  lost 
hospital  and  physician  services  and  the  services  provided  by  the  Nurse 
Practitioner  filled  part  of  that  gap. 

Relationship  Between  Provider  and  Client 

To  a large  extent,  understanding  quality  as  it  relates  to  primary  health  care 
lies  in  understanding  the  relationship  between  provider  and  client.  This 
relationship  is  a complex  one  that  is  influenced  by  a diverse  set  of  factors 
that  projects  have  helped  to  explore  and  enlighten. 


September  2000 


49 


Advancing  Primary  Health  Cara 


Time  and  Support 

An  overriding  theme  of  the  27  primary  health  care  projects  was  that  project 
efforts  should,  ultimately,  make  a positive  difference  in  the  life  of  the 
client/patient.  Whether  focused  on  systemic  issues  or  gathering 
information  about  specific  user  groups  or  implementation  strategies, 
project  proponents  were  oriented  toward  improving  the  quality  of  life  of  the 
client/patient. 

Time  spent  with  the  provider  is  an  important  factor  influencing  clients’ 
perceptions  of  quality.  Several  projects  demonstrated  that  the  time  factor, 
and  the  nature  of  the  interaction  between  provider  and  client  are  critical 
influencers. 


The  best  thing  is  that  the  doctor  spends  time  with  us,  getting  to  know  us.  I think  it 
would  be  nice  if  they  were  open  a couple  of  evening  for  people  who  work.  (Client 
of  the  Alexandra  Community  Health  Centre  (Acton  et  al,  2000). 


In  the  Shared  Mental  Health  project,  for  example,  patients’  levels  of 
satisfaction  with  their  health  care  team  was  very  high.  So,  too,  was  the 
patient’s  level  of  feeling  understood.  The  amount  of  time  spent  with  the 
physician  seemed  to  be  adequate  although  some  patients  would  have 
liked  more  time  to  spend  with  the  mental  health  professional  on  the  team. 

Community  Health  Centre  (CHC)  models  of  service  delivery  in  special 
environments  also  pointed  to  high  levels  of  client  satisfaction  with  the 
quality  of  service  provided.  In  the  CUPS  project  the  collaborative  practice 
model  utilizing  a team  approach,  maintained  the  level  of  quality  service 
clients/patients  experienced  prior  to  the  HTF  project.  Clients  believed  that 
the  Nurse  Practitioner  had  provided  high  quality  care  enabling  the 
physician  to  spend  more  time  with  patients  who  required  that  level  of  care. 
In  addition,  client  perception  was  that  wait  time  had  decreased.  Reduced 
time  to  see  the  care  provider  was  important,  although  other  data 
disconfirmed  this  perception.  This  may,  in  fact,  reinforce  the  perception  of 
improved  quality  of  time  spent  with  the  provider  in  which  case  the  waiting 
time  may  have  seemed  less  significant. 

Similarly,  clients  of  the  Alexandra  CHC  perceived  that  physicians  spent 
more  time  with  them  and  were  more  thorough  than  in  services  provided 
elsewhere.  Clients  saw  care  at  Alexandra  as  “different”  from  that  provided 
in  other  service  venues.  In  particular,  clients  viewed  the  staff  as  more 
caring  and  more  personable  than  elsewhere — again  illustrating  the 
importance  of  patient/provider  relationship  and  perhaps  its  relation  to  time. 

In  the  C.O.P.E.  program,  time  to  collaboratively  discuss  issues  helped  to 
rebuild  relationships  between  school  and  family.  Often, 
miscommunication,  tension,  or  both  had  occurred  between  parents  and 
the  school  over  unresolved  issues  surrounding  a child’s  difficulty.  Parents 
felt  empowered  in  this  process  of  collaboration.  Not  only  were  they  better 
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able  to  identify  and  articulate  their  child’s  problems,  they  felt  part  of  a 
team-based  solution — valued  by  the  input  and  insights  they  were  able  to 
offer.  This  changed  dynamic  in  the  parent/school/medical  triad  pointed 
toward  improved  outcomes  for  children.  Improved  quality  of  care  for 
children  with  emotional,  behavioural  and  learning  problems  resulted. 

Time  at  the  end  stage  of  one’s  life  is  complex  and  ambiguous — too  brief 
and  yet  sometimes  too  difficult  to  endure.  Caregivers  of  palliative  patients 
in  the  Lakeland  Palliative  project  emphasized  the  importance  of  their 
relationship  to  the  Palliative  Nurse  Consultant.  While  beyond  the 
expectations  of  her  assigned  duties,  personal  visits  to  families’  homes  and 
24-hour  telephone  consultation  told  families  they  mattered.  The  fact  that 
physicians  left  telephone  numbers  with  families  (even  when  on  holiday) 
did  not  go  unnoticed.  While  the  support  offered  to  families  was 
appreciated,  the  time  dedicated  to  family  members  at  the  end  stage 
reflected  the  dignity  with  which  health  care  professionals  viewed  the 
client’s  loved  one.  Responding  to  the  naturalness  of  wanting  to  provide  the 
best  quality  care  (and  support  to  caregivers)  reflected  primary  health  care 
in  its  broadest  sense — physical,  mental,  and  spiritual  health  and  well- 
being. 


All  you  had  to  do  was  phone  and  somebody  would  come.  The  program  was 
terrific.  It’s  comforting  to  know  that  there  are  people  out  there  that  strive  to  make  it 
better;  to  bring  death  out  of  the  darkness  and  into  the  warmth  where  it  belongs. 

It’s  time  for  a whole  new  mindset  where  death  is  seen  as  being  normal,  not  as 
being  scary  or  creepy,  but  just  as  being  a natural  part  of  life”  (Howard,  2000). 


For  Somali  women  at  the  Edmonton  Centre  for  Survivors  of  Torture  and 
Trauma,  time  spent  with  the  Cultural  Worker  was  clearly  an  important 
influence  on  their  perspective  of  quality  service.  For  the  women,  being 
able  to  problem  solve  in  a group  setting  and  share  their  fears,  anxieties, 
home-sickness  and  other  experiences  with  women  of  their  own  culture 
was  an  important  part  of  the  settlement  process.  The  value  of  social 
support  provided  by  the  program  was  key. 

For  clients  in  the  Lifestyle  Risk  Reduction  program,  time  spent  with  the 
dietician,  the  fitness  consultant,  and  the  psychologist  as  well  as  the 
support  of  the  cohort  group  (in  addition  to  the  support  of  family  and 
friends)  was  very  important  in  maintaining  client  motivation  to  continuing 
altering  lifestyle  as  an  approach  to  cholesterol  reduction.  Out  of  continued 
motivation  grew  participants’  self-confidence.  Many  expressed  the  belief 
that  they  could  do  what  was  necessary  to  make  lifestyle  changes.  The 
personalized  approach  translated  the  message  from  the  general  mass 
media  and  social  marketing  contexts  into  their  own  personal  experience. 

These  examples  illustrate  the  importance  of  the  client/provider  relationship 
and  its  relation  to  quality  care. 
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Providers’  Perspectives  of  Quality 

Quality  was  also  a significant  theme  among  providers.  While  provider  and 
client  perspectives  on  quality  are  similar  in  some  respects,  providers’ 
views  of  quality  seemed  dominated  by  “quality  control.”  Common 
indicators  of  quality  care  appear  to  be  adherence  to  clinical  practice 
guidelines  and  scopes  of  practice  identified  for  health  professionals,  as 
well  as  quality  improvement  processes. 

Clinical  Practice  Guidelines  and  Scope  of  Practice 

A palliative  sedation  clinical  practice  guideline  (CPG)  was  developed  as 
part  of  the  quality  improvement  process  in  the  Calgary  Palliative  project.  A 
collaborative  and  iterative  process  was  used  to  assess  the  frequency  with 
which  palliative  sedation  was  correctly  administered  (pre-post  tests  and 
chart  audit).  Findings  indicated  that  physicians’  exposure  to  the  tools  used 
to  assess  knowledge  levels  of  palliative  sedation  (as  well  as  past  and 
current  practice  related  to  administering  palliative  sedation),  in 
themselves,  helped  to  raise  awareness  and  knowledge  of  the  palliative 
sedation  CPG,  including  the  need  for  adoption  of  the  CPG  for  improved 
palliative  care  practice. 

Results  of  the  chart  audit  on  palliative  sedation  indicated  a higher  rate  of 
appropriately  sedated  patients  after  dissemination  of  the  CPG  (not 
statistically  significant).  However,  the  rate  of  inappropriately  sedated 
patients  remained  high  (pre  and  post  dissemination).  Data  suggested  that 
the  management  of  delirium  may  be  a significant  factor  that  deserves 
further  study  since  delirium  may  have  played  a role  in  almost  60%  of  all 
cases  reviewed.  (Development  of  a delirium  CPG  is  now  underway.) 

Significant  change  was  reported  in  this  project  with  respect  to  family  and 
team  members’  involvement  in  care  decisions.  Anecdotal  information 
suggested  increased  dialogue  about  palliative  sedation  and  increased 
requests  for  information  about  pain  and  symptom  management.  The 
project  team  understood  the  challenges  associated  with  improving  the 
uptake  of  CPGs.  Alternative  methods  of  dissemination  (in  addition  to  mail- 
outs,  which  are  considered  to  be  a useful  mechanism)  were  used  and 
found  to  be  successful  (e.g.,  discussions,  Grand  Rounds).  Palliative 
clinicians  plan  to  incorporate  the  CPG  into  relevant  consults  and  distribute 
it  to  members  of  the  care  team  as  appropriate.  Together,  these  signs 
indicate  movement  towards  improved  quality  of  care  for  palliative  patients. 

Guidelines  were  also  developed  by  the  staff  at  CUPS  for  ten  medical 
conditions  which  help  to  outline  extended  practice  guidelines  for  the  Nurse 
Practitioner.  Additional  guidelines  will  continue  to  be  developed  and  added 
to  a CUPS  Clinic  Service  Manual.  The  collaborative  practice  model 
utilizing  a team  approach  maintained  the  high  quality  of  service  offered  at 
CUPS.  And,  the  joint  development  of  clinical  practice  guidelines  laid  the 
foundation  for  a continuous  quality  improvement  process.  Care  providers 
at  the  Centre  were  able  to  provide  more  thorough  care  as  a result  of 
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adding  a Nurse  Practitioner  to  the  mix.  Results  indicated  that  95%  of 
CUPS  clients  were  satisfied  with  their  care.  Clinical  practice  guidelines 
were  developed  to  facilitate  the  Nurse  Practioner’s  role  and  these,  along 
with  case  conferences,  promoted  quality  care.  Clients  saw  the  appropriate 
provider  and  made  fewer  repeat  visits  to  the  physician. 


A final  element  that  “worked”  was  that  the  joint  meetings  to  develop  guidelines 
were  held  on  company  time  (Edwards,  2000). 


In  the  Elnora  project,  which  also  involved  the  addition  of  a Nurse 
Practitioner  (NP),  no  new  practice  guidelines  were  developed.  However, 
overall  client  satisfaction  with  services  increased  during  the  course  of  the 
project.  Increases  were  attributed  to  the  personal  attributes  of  the  NP  and 
the  services  she  was  able  to  provide  in  the  absence  of  a physician. 

Quality  Improvement  Processes 

Provider-based  assessment  frameworks  to  assess  quality  are  popular  and 
take  many  forms  (e.g.,  outcomes  management,  physician  profiling, 
continuous  quality  improvement,  benchmarking,  indicators,  severity 
measurements,  appropriateness,  total  quality  improvement).  In  the 
Sunridge  project  (Calgary),  71  quality  indicators  were  identified  and  37 
were  subsequently  selected  in  the  final  evaluation  template  for  a 
continuous  quality  improvement  process  for  primary  care  family  practice 
medical  clinic.  These  indicators  were  rigorously  prioritized  for  relevance. 

As  well,  a manual  on  selecting  an  information  technology  system  for 
primary  care  evaluation  was  developed.  Data  gathered  for  these  indicators 
provide  planners,  physicians,  and  other  staff  with  the  ability  to  gauge  the 
status  of  these  indicators  over  time  and  make  changes  to  health  care 
based  on  results.  Indicators  include  continuity,  comprehensiveness, 
patient  physician  relationship,  and  satisfaction  (see  Stewart,  et.  al.,  2000 
for  a complete  list  of  indicators).  In  this  study  as  well,  an  assessment  of 
asthma  care  was  conducted  to  establish  a baseline  for  later  quality 
improvement  projects.  Areas  for  improvement  were  identified  (e.g., 
increased  use  of  action  plans  to  enable  patients  to  manage  their  own 
care). 


There  is  a very  good  environment  here  for  collaboration  and  openness  to  learn 
from  each  other — one  of  the  strongest  things  about  our  work  here.  There  are  real 
focused  areas  of  expertise  and  good  communication  and  support — Perspective  of 
quality  improvement  collaborator  (Stewart,  Brown,  Harris,  Meredith,  & Hennen, 

| 2000).  


At  the  programmatic  level,  three  independent  initiatives  implemented  in 
the  Calgary  Palliative  project  were  part  of  a Quality  Improvement  Process 
(QIP).  These  initiatives  included 


(a)  identification  of  projects  that  would  result  in  the  enhancement  of 
palliative  care  for  cancer  patients; 

(b)  determination  and  application  of  criteria  for  project  prioritization ; 
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(c)  refinement  of  project  ideas  and  development  of  clearly  defined 
project  statements  for  each  project; 

(d)  implementation  of  projects  that  required  strictly  operational 
decisions; 

(e)  identification  of  project  team  members  including  assignment  of 
tasks  to  design,  implement,  and  evaluate  project  activities; 

(f)  assurance  that  the  project  would  contribute  to  improved  quality  of 
care  by  ongoing  communication  with  the  project  team;  and 

(g)  sponsorship  of  the  project  by  reporting  on  results  and  advocating 
for  adoption  of  recommendations. 

These  components  of  the  QIP  created  an  approach  to  planning  for  quality 
improvement  that  demonstrated  successful  outcomes,  and  its  emphasis 
on  being  "do-able"  within  the  time  frame  of  the  project.  One  specific 
strategy  involved  the  addition  of  the  Patient  Advocate  on  the  primary 
Steering  Committee.  This  inclusion  ensured  that  improvement  was 
constantly  oriented  to  improved  services  for  patients/clients  and  supported 
the  notion  of  client  involvement  in  care  decisions.  The  Leadership  Team 
believed  that  the  QIP  will  be  a useful  mechanism  to  select  future  projects. 

Chart  Audit 


Another  provider-driven  mechanism  to  assess  and  monitor  the  quality  of 
care  is  the  chart  audit.  Several  projects  used  this  method  (e.g.,  asthma 
chart  audit  in  the  Sunridge  project,  Keeweetinok  immunization  chart 
review,  Calgary  palliative,  Lakeland  palliative,  and  CUPS  projects).  Other 
project  findings  illustrated  the  importance  of  the  chart  audit  and  its 
contribution  to  assessing  the  quality  of  services  provided.  The  internal 
evaluation  report  for  the  Alexandra  CHC,  for  example,  recommended 
enhanced  charting  and  file  maintenance  to  help  ensure  quality.  Physicians 
are  in  a process  of  developing  a means  to  self-evaluate  and  measure 
performance.  The  community  development  section  of  the  CHC  is  in  the 
process  of  establishing  outcome  measures  for  community  development 
activities.  In  some  form,  assessment  of  these  activities  will  be  a process 
similar  to  conducting  a chart  audit. 

Collaboration  and  Integration 

Many  projects  provided  opportunities  for  health  providers  to  work  together 
toward  a common  goal.  The  experiences  of  most  of  these  providers 
suggest  that  increasing  understanding  of  each  other’s  practice  results  in 
improved  quality  of  service.  For  example,  in  the  Lakeland  Palliative  project 
physicians  felt  they  provided  better  quality  care  through  improved  pain 
management  control,  expanded  knowledge  about  palliative  care,  and 
increased  capacity  to  provide  improved  access  to  quality  care  for  patients. 
Similarly,  nurses  felt  they  provided  improved  quality  care  as  a result  of 
improved  pain  management  and  coordination  of  care  among  providers 
(achieved  largely  through  improved  communication  and  increased 
interaction  between  home  care  and  acute  care  nurses).  Providers,  as  well 
as  patients,  regarded  the  addition  of  a social  worker  to  the  provider  team 
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as  an  important  element  in  improving  the  quality  of  care  for  palliative 
patients.  Home  care  nurses  also  expressed  increased  skill  in  pain 
assessment,  knowledge  about  palliative  care,  pain  control,  including 
knowledge  about  the  social  needs  of  families  and  patients  during  and  at 
end  stage  of  illness. 

Medical  team  members  involved  in  diagnosing  children’s 
emotional/behavioural  and  mental  health  problems  in  the  C.O.P.E.  project, 
demonstrated  that  moving  into  an  atypical  practice  environment  forced 
psychiatrists,  developmental  pediatricians,  and  psychologists  to  expand 
the  circle  of  people  involved  in  contributing  to  diagnosis  of  children’s 
problems.  The  Family  Liaison  and  project  coordinator  expressed  that  they 
had  learned  a great  deal  about  medical  terminology  associated  with 
children’s  mental  health  problems.  As  a professional  development 
opportunity,  collaborative  consultation  had  increased  their  capacity  to 
assist  parents  and  school-based  staff  in  addressing  emotional  and 
behavioural  needs  of  children.  Similarly,  scopes  of  practice  for  school- 
based  professionals  (Learning  Specialists,  teachers/administrators)  also 
seemed  to  change  as  a result  of  their  expanded  repertoire  and  skill  at 
identifying  and  articulating  children’s  mental  health  problems.  The  project 
also  demonstrated  that  some  physicians  exhibited  greater  proclivity  to 
working  in  non-medical  (i.e.,  school)  environments  than  did  others. 

Primary  health  care  environments,  then,  may  call  on  medical  professionals 
to  move  beyond  their  normal  practice  patterns. 

The  shared  care  model  in  the  Shared  Mental  Health  project  demonstrated 
job-related  changes  for  both  physicians  and  mental  health  clinicians  that 
seemed  to  influence  their  perspectives  of  the  level  of  quality  care  they 
were  able  to  provide.  For  example,  physicians  reported  greater 
effectiveness  and  confidence  in  dealing  with  mental  health  issues,  fewer 
and  more  appropriate  referrals  to  external  agencies,  improved  diagnostic 
capabilities,  more  time  spent  counseling  patients,  and  greater  attention 
and  time  dedicated  to  studying  mental  health  issues.  Mental  health 
clinicians  indicated  that  physicians  shifted  from  being  solely  “prescribers” 
and  “fixers”  to  being  enablers  and  facilitators,  and  seemed  to  prescribe 
psychotropic  medication  more  appropriately.  The  psychiatrists  felt  that  the 
challenges  of  consultation-liaison  psychiatry  in  primary  care  made  their 
jobs  more  interesting  in  terms  of  the  skills  required.  One  of  the  challenges 
they  encountered  was  synchronizing  their  schedules  with  those  of  the 
family  physicians. 

Both  Healthy  Families  projects  (Capital  Health/urban  and  WestView 
region/rural)  demonstrated  the  benefit  of  training  Family  Workers  to 
appropriately  interface  with  young  mothers  and  their  babies.  A train  the 
trainer  model  was  used  to  share  learning  from  lead  Family  Workers  to 
others,  which  offset  the  costs  of  travel  to  the  U.S.  Projects  documented 
the  importance  of  certified  trainers  to  be  able  to  deliver  quality  care  and 
support  to  mothers  and  their  children. 
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Role  Clarification  and  Perception  of  Quality 

One  of  the  interesting  findings  with  respect  to  perceptions  of  quality 
related  to  what  providers  thought  their  own  and  other  providers’  roles  were 
in  the  cycle  of  care.  The  360-degree  mechanism  used  to  assess 
perspectives  of  providers,  patients,  and  patients’  families  in  the  Calgary 
Palliative  project  indicated  assumptions  about  what  providers  thought  they 
should  know  to  be  able  to  provide  quality  care,  but  what  patients  thought 
they  should  know  and  do.  These  priority  needs  are  shown  in  Table  1. 


Table  1.  Practitioner  Priority  Needs 
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about  local 
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medicine 
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Findings  also  indicated  that  families  believed  that  the  level  of  interpersonal 
skill  of  oncologists  could  be  improved  with  respect  to  imparting  diagnostic 
information  to  patients,  and  general  information  about  palliative  care  to 
family  members.  Families  wanted  and  expected  information  about 
alternative  and  complementary  therapies.  It  follows,  logically,  then  that 
when  priority  needs  are  met,  perceptions  of  quality  are  likely  to  be  more 
favourable. 


Changing  Perceptions  of  Quality 

Quality  is  not  static.  It  changes  with  perception.  A recurring  theme  across 
projects  was  that  increased  information  led  to  increased  knowledge,  and 
increased  knowledge  improved  understanding.  With  improved 
understanding  both  clients  and  providers  were  better  able  to  assess 
quality.  In  this  manner,  as  a user  becomes  a “connoisseur”  (Eisner,  1986), 
judgment  and  expectation  of  quality  changes. 

Client  Information 

Specific  examples  illustrate  that  a social  marketing  approach  to  building 
public  awareness  of  specific  practices  within  a primary  health  care 


environment  is  important.  The  Sedation  Team  (Calgary  Palliative  project) 
reinforced  the  importance  of  continuing  to  advocate  at  the  national  level 
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that  “the  philosophy  of  palliative  care  and  appropriately  utilized 
interventions  such  as  palliative  sedation  are  ethical  and  legal.”  The 
Northeast  Health  Centre  “targeted”  approach  required  that  the  community 
needed  to  be  informed  about  the  scope  and  purpose  of  the  Centre 
(Brochures  were  produced.) 

The  Misericordia  Health  Lifestyle  project  chose  a media  release  approach 
that  generated  more  inquiries  than  could  be  handled  by  the  project. 
However,  response  to  the  media  campaign  gave  the  team  confidence  that 
public  interest  was  very  high  concerning  alternative  methods  to  controlling 
cholesterol.  Participants  reported  increased  knowledge  and  improved 
understanding  of  the  relationship  between  readiness  to  take  action  and 
taking  action.  In  gaining  an  appreciation  of  what  the  course  helped  them  to 
understand  (e.g.,  the  nature  of  cravings  and  how  to  control  them), 
judgments  about  subsequent  courses  on  lifestyle  change  and  risk 
reduction  will  likely  be  viewed  differently.  Similarly,  clients  at  the  Edmonton 
Centre  for  Survivors  of  Torture  and  Trauma  may  expect  more 
sophisticated  services  and  social  supports  in  new  programs.  As  their  level 
of  assimilation  into  Canadian  society  is  extended,  clients’  expectations  and 
therefore  judgments  of  quality  of  programs  beyond  settlement  outcomes 
may  change. 


“Here  111  give  you  one  of  my  cholesterol  pills. . .my  doctor  gave  them  to  me,  now  I 
can  eat  anything  I want.  ” (Focus  group  participant  commenting  on  poor  public 
information)  (Ausford,  Wheeler  and  Wanek,  2000). 


Provider  Information 
Baseline  Information 

The  majority  of  projects  emphasized  that  collection  of  baseline  information 
and  informatics  play  a critical  role  in  the  ability  to  assess  quality.  The  need 
for  timely,  relevant,  and  accurate  information  cannot  be  overstated. 

Communication  Among  Providers 

Information  about  a patient’s  condition  is  the  foundation  to  providing 
quality  care.  The  Calgary  Palliative  project  demonstrated  that  a 
standardized  consultation  note  led  to  timely  transfer  of  information 
between  oncologists  and  family  physicians.  Physicians  considered  the 
template  an  improvement  over  previous  reporting  formats  in  that  it  allowed 
for  easy  access  to  relevant  and  timely  information  about  care  prescribed 
by  oncology  specialists.  Since  the  Tom  Baker  Cancer  Clinic  provides  care 
to  most  of  southern  Alberta,  communication  among  providers  is  key.  From 
the  perspective  of  oncologists,  the  template 

(a)  increased  timeliness  of  dictations  received  by  family  physicians, 

(b)  increased  organization  of  the  content  of  letters  included  in  patients’ 
files,  and 

(c)  had  applicability  beyond  cancer  patients. 
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Nurses  at  the  Lung  Clinic  supported  this  finding  in  that  nurses  said  they 
found  transcribing  oncologists’  dictations  relatively  easy  using  the  template 
format.  Beginning  in  the  fall  of  2000,  the  template  will  be  used  for  all 
patients  in  the  Lung  Clinic.  Oncologists  also  expressed  a desire  to  use  the 
template  in  other  clinics  as  well  (e.g.,  Neurology  Clinic).  They  suggested 
that  the  template  could  be  tailored  for  more  specific  use  (e.g.,  for  follow 
up,  clinical  trials)  and  that  it  could  be  used  with  non-palliative  patients. 
They  felt  that  some  clarification  of  use  of  terms  was  required  (e.g., 
“potential  problems”)  and  emphasized  the  need  to  maintain  patient 
identifiers. 

The  level  of  satisfaction  with  telephone  communication  between  family 
physicians  and  oncologists  was  also  investigated  in  the  Calgary  Palliative 
study.  Family  physicians  reported  satisfaction  with  the  amount  of 
information  received  once  appropriate  contact  was  made.  However,  they 
also  reported  difficulty  making  that  initial  contact.  Physicians  offered 
several  suggestions  for  improved  telephone  communication — most  of 
which  pointed  to  a dedicated  phone  line — one  of  the  original  objectives 
identified  by  the  Communication  Team. 

Several  other  projects  illustrated  the  importance  of  shared  information 
among  providers.  The  Sunridge  project  depended  on  collaborative 
communication  to  establish  indicators  of  quality.  The  Lakeland  Palliative 
project  was  centred  on  quality  care  through  interdisciplinary 
communication  among  nurses,  physicians,  and  pharmacists. 

Communication  Between  Provider  and  Client/Patient 

The  C.O.P.E.  project  demonstrated  that  increased  accurate  information 
from  parents  allowed  physicians  to  make  more  informed  decisions  about 
diagnoses  of  children’s  mental  health  problems.  Through  increased 
dialogue  and  collaborative  discussion 

(a)  earlier  and  comprehensive  diagnosis  was  achieved, 

(b)  teachers  had  increased  ability  to  present  and  articulate  children’s 
mental  health  problems  (which  facilitated  more  appropriate 
diagnosis  and  thus  improved  the  quality  of  care  for  children  with 
mental  health  problems), 

(c)  parents  in  the  study  group  increased  their  ability  to  identify 
emotional  and  behavioural  problems  more  than  did  parents  in  the 
control  group,  and 

(d)  study  group  parents  became  more  knowledgeable  about  resources 
available  to  them  and  were  more  likely  to  access  them  than  were 
the  control  group  parents. 

An  interesting  sub-theme  of  communication  is  the  mediation  role  that 
medical  team  members  (psychiatrist,  developmental  pediatrician, 
psychologist)  often  played  between  school  and  family. 
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The  framework  for  quality  used  in  the  evaluation  of  the  Elnora  Health 
Centre  project  was  based  on  the  assumption  that  community  participation 
and  group  processes  which  make  beneficial  changes  for  individuals  and 
the  community  are  in  themselves  health  enhancing.  This  framework  is 
based  on  the  Canadian  Council  on  Health  Services  Accreditation  model 
which  focuses  more  on  professional  and  managerial  behaviour  than  on 
client  satisfaction.  However,  both  tools  emphasize  meeting  client  and 
community  expectations  and  working  in  partnership. 

Relations  between  Metis  representatives  and  Lakeland  Regional  Health 
Authority  have  improved.  Anecdotal  evidence  suggests  that  the  presence 
of  the  Settlement  Nurse  is  bringing  health  to  the  forefront  and  supporting 
more  people  to  take  action  to  improve  their  health.  Community  members 
have  come  to  understand  that  the  Settlement  Nurse  can  help  them  with 
health  issues  and  provide  alternatives  for  dealing  with  these  issues. 

Support  offered  to  at-risk  women  in  the  Healthy  Families  projects  was  also 
evidence  of  the  importance  of  establishing  a relationship  between  provider 
and  client. 

Measuring  Quality 


Anecdotal  versus  Standardized  Measurement  of  Quality 

Most  evidence  for  client  satisfaction  of  care  provided  by  projects  was 
anecdotal  in  nature.  Some  exceptions  to  this  included  the  Shared  Care 
project  in  which  quantitative  data  supported  conclusions  from  the  open- 
ended  data  that  the  shared  care  model  was  perceived  by  patients  and  by 
the  team  of  health  professionals  in  very  positive  terms.  The  SF-36  Mental 
Health  and  Physical  Health  summary  scores  were  correlated  with  the 
Patient  Satisfaction  Scale  and  the  Patient  Enablement  Scale.  Correlations 
were  non-significant  with  the  former,  but  significant  with  the  latter. 

In  the  C.O.P.E.  project  questions  were  drawn  from  the  National 
Longitudinal  Survey  of  Children  and  Youth  (a  standardized  instrument)  to 
gather  information  about  parents’  perspectives  about  the  behaviour  of 
children  (an  indirect  indication  of  the  influence  of  the  intervention  or 
quality). 

The  Collectives  project  demonstrated  innovative  use  of  three  instruments: 
(a)  the  Medication  Appropriateness  Index  in  a community  setting,  (b)  the 
Health  Utilities  Index  in  a primary  care  community  setting,  and  (c) 
extended  use  of  an  instrument  to  measure  collaboration  and  satisfaction 
about  care  decisions.  The  independent  evaluator  suggested  that 
opportunity  exists  to  develop  a more  appropriate  instrument  to  assess  the 
quality  of  prescribing  in  primary  care  (the  right  medication  at  the  right 
dosage). 
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The  Northeast  is  currently  in  the  process  of  developing  an  appropriate 
client  satisfaction  instrument  based  on  an  extensive  review  of  the  literature 
and  revision  of  a standardized  instrument  to  suit  local  context  and  need. 

So  far,  project  participants  have  found  the  exercise  a resource  intensive 
and  context  specific  experience.  The  project  discovered  that  more  time  is 
required  to  evaluate  outcome  measures  in  a new  CHC  and  that 
benchmarks  are  difficult  to  find. 

Planning  for  Quality  Service 

Several  projects  demonstrated  that  strategic  planning  to  improve  quality 
can  occur  at  regional,  program,  and  delivery  levels. 

Model  Development 

In  the  Calgary  Regional  Health  Authority  the  Repositioning  project 
explored  a Total  Health  Process  (THP).  Tools  associated  with  the  process 
hold  potential  to  facilitate  improved  quality  of  services  (i.e.,  mapping  to 
identify  gaps  in  services,  “game”  playing  to  build  consensus  around  a 
reconfigured  model  of  service  delivery,  and  population  modeling  to  match 
resources  to  population  needs).  According  to  the  model,  patients  are 
managed  through  multidisciplinary  teams — primary  health  care  anchors, 
which  are  responsible  for  managing  the  care  of  patients  across  the 
continuum  through  capitated  budgets. 


The  project  team  suggests  that  as  patients  self-select  an  appropriate  anchor  and 
team,  and  as  their  funding  goes  with  them,  a market  dynamic  will  be  created 
which  would  encourage  providers  to  provide  quality  care  (Church,  Gerlock,  La 
Fleur,  & Mark,  2000). 


Independent  evaluators  caution  that  this  can  happen  only  if  resources  are 
reallocated  to  fill  gaps  in  the  service  continuum  and  support  new 
processes,  service  configurations,  and  populations  with  highest  need. 

They  caution  further  that  if  the  model  is  intended  to  move  toward  an 
integrated  delivery  system,  "existing  research  does  not  clearly  support  the 
proposition  that  increased  integration  leads  to  improved  quality  of  service" 
(Church  et  al.,  2000).  Because  full  implementation  of  the  model  did  not 
occur,  it  is  not  possible  to  determine  what  impact  the  model  might 
ultimately  have  on  quality  of  service. 

Community  Consultation 

In  the  East  Central  Regional  Health  Authority  a process  was  undertaken  to 
plan  for  the  implementation  of  a primary  health  services  system.  While  the 
process  was  piloted  at  four  sites  (communities),  the  intention  is  to 
eventually  implement  a primary  health  delivery  system  throughout  the 
region.  In  this  process,  regional  staff  worked  with  communities  through  a 
consultation  process  to  identify  and  address  quality  issues.  The 
consultation  process  involved  gathering  community  perceptions  of  access 
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and  service  gaps  in  the  four  communities,  as  well  as  community  members’ 
satisfaction  with  the  process  used  by  the  region  and  health  providers  to 
deal  with  health  concerns.  This  process  was  regarded  as  a precursor  to 
planning  for  quality  health  services  in  the  region. 

Targeting  Specific  Areas 

Learning  evolved  from  the  continuous  quality  improvement  process 
implemented  in  the  Sunridge  project  including 

1.  incentives  and  resources  are  required  to  incorporate  quality 
management  activities  at  the  local  level, 

2.  capacity  to  do  quality  work  at  the  local  level  must  be  enhanced, 

3.  racilitating  stakeholder  participation  in  the  selection  of  indicators  helps 
to  ensure  a successful  evaluation  process, 

4.  local  factors  necessitate  adaptation  of  existing  tools, 

5.  input  from  clinic  staff  enhances  the  collection  of  reliable  data, 

6.  international  and  national  benchmarks  require  local  validation, 

7.  patient  care  needs  must  take  precedence  over  the  need  to  gain 
information,  and 

8.  patient  inflexibility  influences  the  clinic’s  ability  to  meet  their  needs. 

Evaluation  of  service  delivery  at  the  8th  and  8th  Health  Centre  in  Calgary, 
which  provides  24-hour  service,  was  useful  in  identifying  specific  areas 
that  need  to  be  addressed  to  improve  quality.  These  include 

(a)  the  need  to  clearly  define  the  target  population  with  a focus  on  the 
vulnerable  or  marginalized; 

(b)  safety  concerns  for  clients  and  staff,  particularly  receptionists  who 
might  be  exposed  to  potential  violence  from  mentally  unstable  or 
inebriated  individuals; 

(c)  concern  that  the  appearance  of  some  clients  caused  other  clients 
to  seek  care  elsewhere;  and 

(d)  lack  of  privacy  in  the  reception  area. 

Similarly,  results  of  the  evaluation  of  the  Alexandra  Community  Health 
Centre  revealed  the  necessity  to  revisit  services  provision  in  light  of  the 
changing  demographic  of  community  members. 

Summary  of  Findings 


1 . Clients  play  a major  role  in  determining  what  constitutes  quality 
primary  health  care.  Client  perceptions  of  health  service  quality  are 
closely  related  to  perceptions  of  access  to  health  services.  To  a large 
extent,  understanding  quality  as  it  relates  to  primary  health  care  lies  in 
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understanding  the  relationship  between  provider  and  client.  Often  a 
positive  relationship  is  defined  by  the  amount  of  time  a provider 
spends  with  a patient/client. 

2.  Providers  tend  to  take  a more  technical  perspective  in  assessing 
quality.  They  relate  quality  to  prescribed  standards  set  through  clinical 
practice  guidelines,  identification  of  measurable  quality  indicators, 
institution  of  quality  improvement  processes,  and  chart  auditing. 

3.  Not  only  is  quality  primary  health  care  linked  to  access,  it  appears  to 
have  a reasonably  strong  association  with  integration  of  information 
between  service  providers.  Providers  acknowledge  that  quality  of 
services  can  be  improved  through  broadening  their  scopes  of  practice, 
collaborating  with  other  health  professionals,  and  role  clarification. 
Some  evidence  suggests  that  information  integration  can  be  facilitated 
through  standardized  communication  templates,  and  common  medical 
records. 

4.  The  definition  of  quality  primary  health  care  changes  with  individual 
perceptions  of  quality.  Perceptions  change  through  changes  in 
knowledge  and  understanding.  This  applies  to  both  clients  and 
providers.  Through  more  information  clients  begin  to  realize  that  they 
have  certain  responsibilities  in  maintaining  and  improving  their  health. 
Providers  better  understand  what  constitutes  quality  service  through  a 
greater  appreciation  of  what  colleagues  and  other  practitioners 
perceive  as  quality. 

5.  Changing  perceptions  of  quality  make  it  difficult  to  measure  quality. 
While  standardized  instruments  are  available,  they  often  require 
modification  to  fit  specific  primary  health  care  environments.  However, 
qualitative  indicators  of  quality  are  extremely  important  in  recognizing 
the  “personal”  impact  of  services  on  clients. 

6.  Mapping  exercises,  model  development,  community  consultation  and 
continuous  quality  improvement  processes  can  assist  in  identifying  the 
changing  needs  of  a population,  thereby  contributing  important 
information  to  strategic  planning  processes  for  the  delivery  of  quality 
health  services. 

7.  Challenges  to  improving  quality  of  primary  health  care  include 

(a)  difficulty  identifying  needs  of  the  under-served  and/or  marginalized, 

(b)  inability  to  aggregate  data  for  certain  populations  (e.g.,  Metis), 

(c)  time  constraints  of  physicians  and  other  health  professionals, 

(d)  willingness  to  share  information  across  sectors, 

(e)  deficient  information  systems, 

(f)  sampling  techniques  (e.g.,  exclusion  of  specific  audiences  such  as 
persons  with  mental  illness),  and 

(g)  shifting  from  episodic  care  to  establishing/building  provider/patient 
relationship. 


September  2000 


62 


Advancing  Primary  Health  Care 


Impact  on  Populations 


Health  status  is  intended  to  serve  as  an  indication  of  the  degree  of  health 
present  among  individuals  and  populations.  While  impact  on  populations 
was  identified  as  a key  dimension  of  primary  health  care  for  the  purposes 
of  the  Health  Transition  Fund  projects,  most  of  the  Alberta  projects  did  not 
include  a change  in  relative  health  of  a population  in  their  set  of  objectives. 
Either  the  projects  did  not  emphasize  impact  on  populations  as  a primary 
health  care  issue  or  the  two-year  time  frame  for  implementation  of  projects 
was  insufficient  to  implement  interventions  and  assess  their  relation  to 
population  health  status.  However,  strategies  were  employed  with  the 
expectation  that  learning  from  the  projects  would  eventually  contribute  to 
positive  impact  on  populations.  A smaller  number  of  projects  had  clearly 
articulated  objectives  and  measures  in  place  to  assess  impact. 

Measurement  of  population  health  has  always  been  a critical  issue,  and  a 
great  deal  of  effort  has  gone  into  attempts  to  quantify  the  health  of 
populations.  In  the  Alberta  projects,  health  status  indicators  were  largely 
used  for  strategic  planning  purposes  and  as  measures  of  program 
success. 

Impact  on  Population  Health  Status  and  Strategic  Planning 

In  order  to  measure  changes  in  the  health  of  populations,  baseline 
information  must  be  available.  Epidemiological  information  can  then  be 
used  to  inform  organizational  strategic  planning.  At  the  Northeast 
Community  Health  Centre,  the  Regional  Health  Status  Report  (Capital 
Health  Authority)  is  used,  along  with  community  consultation,  to  determine 
which  target  populations  are  most  in  need  of  health  services.  The  Report 
publishes  health  status  outcome  measures  by  segments  of  the  regional 
population  on  various  indicators  such  as  birth  weight,  heart  disease, 
diabetes,  teen  smoking,  and  childhood  injuries.  These  data  were 
instrumental  in  the  selection  of  family  health,  women’s  health,  community 
health,  mental  health,  and  child  and  adolescent  health  as  primary  service 
areas. 


Supporting  healthy  lifestyles  and  optimizing  health  status  are  central  to  the  overall 
vision  and  mission  of  the  NECHC  and  guide  daily  activities  (Wilson,  2000). 


Other  projects  adopted  different  approaches  to  address  impact  on 
population  health  status.  The  approach  taken  by  the  Immunization  project, 
for  example,  had  a definite  impact  on  improving  regional  immunization 
rates.  Baseline  data  indicated  that  the  region  had  low  immunization  rates. 
Following  implementation  of  the  first  intervention  strategy  (Permission  to 
Contact  Letter),  37%  of  the  target  group  made  a change  in  their 
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immunization  status.  An  additional  26%  made  a change  after  the  second 
implementation  strategy  (a  telephone  interview).  These  results  will  help 
the  region  to  plan  future  immunization  strategies.  Also,  since  immunization 
rates  serve  as  a proxy  measure  of  health  status,  this  information  serves  as 
an  indicator  of  population  health  in  the  region. 

Population  Health  Status  as  an  Indicator  of  Program  Success 

Strategies  aimed  at  improving  population  health  status  often  focus  on 
providing  services  to  groups  of  patients  with  common  chronic  diseases 
(e.g.,  diabetes,  asthma).  Health  status  indicators  (appropriate  for  these 
diseases)  have  utility  as  indicators  of  success  for  specific  programs  aimed 
at  addressing  these  diseases.  These  indicators  also  enable  comparison  to 
larger  samples  of  the  population.  In  addition,  health  status  indicators  can 
serve  as  markers  for  quality  control. 

Marker  of  Quality 

The  study  of  the  adult  asthma  care  delivered  by  the  Sunridge  family 
practice  teaching  clinic  in  Calgary  was  undertaken  to  identify  those 
patients  who  had  asthma  and  to  assess  their  care  from  a quality 
perspective.11  Secondly,  as  an  ambulatory  care  sensitive  condition,  the 
study  was  undertaken  as  a marker  of  quality  and  utilization  (although  the 
authors  note  that  this  latter  purpose  has  been  challenged  in  the  literature). 
Responses  to  the  Asthma  Quality  of  Life  Questionnaire  (AQLQ)  and  the 
SF-36  Survey,  as  well  as  an  in-depth  chart  assessment12  indicated 
potential  areas  for  improvement  in  care  including 

(a)  increased  use  of  objective  measurement  in  cases  where  asthma 
was  the  reason  for  the  encounter, 

(b)  increased  use  of  spacer  devices, 

(c)  increased  use  of  action  plans  to  enable  patients  to  manage  their 
own  care, 

(d)  the  possibility  that  some  patients  are  not  receiving  appropriate 
inhaled  steroid,  and 

(e)  a possibility  to  reduce  smoking  and  smoking  recording  among 
asthmatics. 

Standardized  Self-Reported  Measurements 

A standardized  instrument,  the  Short  Form-36  (SF36),  was  used  in  the 
Shared  Care  Mental  Health  project  as  a measure  of  patient-perceived 
health  status.  Post-treatment  SF-36  data  were  compared  with  normative 
data  from  two  groups  (a  group  of  chronically  ill  patients  who  were  relatively 


11  Note:  The  Sunridge  project  was  not  focused  on  health  status.  However,  over  time  the  health  status 
indicators  chosen  for  the  clinic  will  be  an  important  component  of  the  continuous  quality  improvement 
process. 

12  The  chart  assessment  examined  demographics,  medication  usage,  attendance,  smoking  status, 

asthma  care,  co-morbidity,  consultations,  and  emergency  utilization. 
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well  and  a group  with  psychiatric  conditions  such  as  depression). 

Following  treatment,  the  shared  care  group  (i.e.,  the  study  sample) 
exhibited  scores  between  the  scores  (in  the  middle)  of  the  other  two 
comparison  groups.  Pre-post  SF-36  scores  were  available  for  a small 
group  (n=18)  of  the  study  sample.  These  latter  scores  were  higher  at  post- 
treatment suggesting  they  may  have  moved  toward  improved  health  status 
from  pre-treatment  levels. 

The  Collectives  project  also  used  health  status  indicators  to  measure  the 
success  of  their  intervention  (collaborative  case  management).  While 
health  status  did  not  significantly  improve,  it  was  maintained  from  baseline 
to  follow  up  with  no  differences  across  Collectives.  The  study  concluded 
that  improved  access  to  and  coordination  of  pharmacy  and  home  care 
services  with  family  practice  services  for  people  with  complex  health 
needs  significantly  improves  medication  compliance.  In  this  project,  of  the 
240  issues  identified  by  the  Collectives,  25%  related  to  decline  in  health 
status.  Impact  on  this  generally  unwell  group  was  maintained  from 
baseline  to  three-  and  six-month  follow  up.  No  statistically  significant 
difference  in  overall  functional  ability  and  health  status  was  reported 
(vision,  hearing,  speech,  ambulation,  dexterity,  emotion,  cognition,  pain). 
Two  standardized  tests  of  health  status  were  used:  Health  Utilities  Index- 
HUI  and  SF-12.  The  researchers  caution  that  since  there  was  no  control 
group  in  the  design  of  this  innovative  and  exploratory  project,  it  is  not 
possible  to  draw  conclusions  that  would  confirm  that  any  changes  were 
the  direct  and  sole  result  of  the  intervention  by  the  Collectives. 
Nevertheless,  these  data  serve  as  useful  baseline  information  for  future 
comparison. 

Laboratory  Measurement  of  Health  Impact 

Serum  cholesterol  levels  of  study  participants  were  used  as  an  indicator  of 
health  status  in  the  Lifestyle  Risk  Reduction  project.  Pre-post  measures 
indicated  that  both  the  treatment  and  the  control  groups  decreased  overall 
levels.  (No  statistically  significant  differences  in  effect  could  be  reported.) 
Overall,  both  treatment  and  control  group  clients  demonstrated  decreased 
serum  cholesterol  and  increased  frequency  of  physical  activity.  Forty-three 
percent  (43%)  of  participants  in  the  treatment  group  and  35%  in  the 
control  group  achieved  their  target  LDL  cholesterol  (Low  Density 
Lipoprotein).  Treatment  group  clients  demonstrated  some  improved 
dietary  habits.  Statistical  results  must  be  interpreted  with  caution, 
however,  as  very  few  participants  were  involved  in  the  study.  Long-term 
outcome  measures  are  required  to  assess  sustained  changes  over  time.  A 
number  of  extraneous  variables,  such  as  the  following,  require 
examination  before  a definitive  conclusion  can  be  drawn  concerning  the 
effect  of  the  intervention: 

• influence  of  the  Hawthorne  effect  (i.e.,  the  implication  of  the  “effect  of 
being  tested”  on  subjects), 

• differences  in  treatment  and  control  group  cholesterol  levels  at  the 
beginning  of  the  study, 
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• short  duration  of  the  study, 

• advanced  stage  of  contemplation  of  both  control  and  treatment  groups, 
and 

• referent  power  of  physicians  over  the  short-term. 

Client  Self-Reported  Health  Impact 

A number  of  projects  used  non-standardized  approaches  to  determine 
program  impact  on  health  status.  The  HA5  project,  for  example,  was  able 
to  collect  pre-post  data  on  health  attitudes  and  behaviours  including 
client/patient  self-perceived  health  status.  Since  data  were  drawn  from  a 
randomized  sample  of  the  population  in  four  rural  communities  in  the 
region,  results  can  serve  as  baseline  information  (region-wide).  Results 
indicate  interesting  health  impacts  reported  as  a result  of  introducing  multi- 
disciplinary health  teams  to  the  four  rural  communities. 

1 . Evidence  appears  to  support  the  contention  that  multidisciplinary,  rural 
health  services,  and  community  development  activities  can  have  a 
positive  impact  on  health  attitudes  and  behaviours  (i.e.,  higher 
proportion  of  people  in  the  region  seeking  health  information  or  having 
received  health  information,  more  people  agreeing  that  drinking 
alcohol  reduces  health  status,  general  decline  in  smoking,  higher 
proportion  of  people  trying  to  increase  the  amount  of  fruit  and 
vegetables  in  their  diets,  increased  numbers  of  people  having  their 
blood  pressure  checked). 

2.  Communities  reported  an  increase  in  use  of  the  internet  as  a means  to 
find  health-related  information. 

3.  Treatment  communities  reported  a significant  increase  in  taking 
prescribed  medication  and  receiving  flu  shots,  whereas  the  control 
community  did  not. 

While  not  conclusive,  evidence  suggests  that  multidisciplinary  rural  health 
services  and  community  development  activity  have  a positive  impact  on 
self-perceived  health  knowledge.  Over  time,  these  baseline  data  may  be 
useful  in  determining  how  communities  change  their  self-perception  of 
physical  and/or  mental  health  status. 

The  following  table  identifies  other  projects  that  used  self-reported,  non- 
standardized  data  as  an  indicator  of  health  impact. 
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Table  2.  Projects  Using  Self-Reported  Indicators  of  Health  Impact 


Projects 

Self-Reported  Health  Impact 

C.O.P.E. 

After  a 4-month  intervention,  study  group  parents 
perceived  improved  emotional/behaviour  in  their 
children. 

Collectives 

Patient  self-reported  data  on  health  services  utilization 
suggested  a trend  toward  fewer  physician  visits, 
emergency  and  hospital  stays 

Shared  Mental 

Close  to  70%  of  the  patients  interviewed  felt  that  they 

Health 

had  shown  some  improvement.  The  majority  of  patients 
felt  that  the  intervention  improved  their  ability  to  cope 
with  life,  understand  their  illness,  cope  with  their  illness, 
stay  healthy,  have  increased  confidence  about  health, 
and  be  able  to  help  themselves.  Over  60%  of  the 
patients  were  able  to  describe  their  symptom/illness 
from  a bio-psychosocial  perspective.  Patients  also 
commented  on  the  convenience,  accessibility,  and 
absence  of  mental  health  stigma  with  being  treated  in 
the  physician’s  office. 

Diabetes  Education 

After  attending  the  program,  the  percent  of  clients  who 
indicated  that  their  blood  sugar  levels  were  out  of  control 
decreased  from  64%  to  22%. 

Provider-Reported  Health  Impact 

Data  gathered  from  providers  about  patient/client  health  status  generally 
involved  anecdotal  comments.  For  example,  in  the  CUPS  project,  while  no 
significant  difference  in  health  status  of  12  clients  was  reported  using  pre- 
post assessments,  clinic  staff  believed  that  the  health  status  of  these 
clients  had  improved. 

Of  clients  served  by  mental  health  staff  at  the  8th  and  8th  Health  Centre  in 
Calgary  in  1999,  87%  scored  50  or  greater  on  the  Mental  Health 
Diagnostic  Scale,  which  generally  suggests  that  they  required  hospital 
care.  These  findings  indicate  that  hospitalization  might  have  been  averted 
for  a significant  number  of  mental  health  clients  accessing  Centre  services 
and  that,  with  support,  these  individuals  could  continue  to  be  served  in  the 
community. 

Findings  reported  by  the  psychologist  who  counseled  survivors  of  torture 
and  trauma  who  were  participants  in  the  clinical  program  at  the  ECSTT 
indicated  that  clients  experienced  improved  physical  and  mental  well- 
being (e.g.,  sleeping  better,  reduced  anxiety,  fewer  marital  problems, 
being  able  to  focus,  improved  memory).  In  addition  these  case  study 
reports,  the  Hopkins  Symptom  Checklist  25  was  used  at  pre-  and  post- 
treatment. 
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Health  Impact  and  Healthy  Families 

Because  primary  health  care  considers  health  very  broadly,  a variety  of 
developmental,  social,  and  emotional  measures  are  appropriate  to  assess 
impact  on  populations.  This  was  demonstrated  in  the  Healthy  Families 
Capital  project  where  fewer  pre-term  and  low  birth  weight  babies  were 
reported.  In  addition,  59%-80%  babies  were  reported  to  be  breast  fed,  up- 
to-date  immunizations  were  reported  for  approximately  90%  of  babies,  and 
80%  of  children  were  reported  within  the  normal  range  for  healthy 
development.  The  WestView  project  reported  97%  of  children  within  the 
normal  range  for  healthy  development.  (The  Denver  Developmental 
Screening  Test  II  was  used  in  both  projects.) 


After  18  months  of  implementation  activities  and  operation,  the  Healthy  Families 
Program  in  the  Capital  Health  Region  is  already  demonstrating  positive  impacts 
(C.A.  MacDonald  and  Associates,  2000). 


Of  all  projects,  the  Healthy  Families  Capital  project  used  the  greatest 
number  of  tools  to  measure  program  impact  on  health  status  as  a result  of 
a primary  health  care  intervention.  Instruments  included:  The  Family 
Stress  Checklist,  Family  Assessment  Device,  Maternal  Social  Support 
Index,  Adult-Adolescent  Parenting  Inventory,  Child  Development 
Inventory,  Centre  for  Epidemiological  Studies  Depression  Scale,  Carey 
Infant  Temperament  Scale,  and  Home  Observation  for  Measurement  of 
the  Environment  Scale.  Some  of  the  significant  findings  included  the 
following. 

1 . Parents  have  increased  knowledge  and  use  of  community  supports. 
(Clients  value  the  important  connections  that  have  been  provided  to 
them  by  the  agency.  Agencies  are  using  a number  of  mechanisms 
such  as  direct  referrals  to  other  agency  programs  offering  daycare, 
group  activities,  and  introduction  to  other  staff  members.) 

2.  Parents’  personal  development  appeared  to  improve  in  some  areas 
and  deteriorate  in  others.  (Parents  scored  high  on  a scale  measuring 
parental  knowledge  of  four  key  areas  of  child  development:  emotional, 
cognitive,  physical,  and  social.  Scores  ranged  from  73  to  92  (maximum 
score  100).  Knowledge  of  social  development  was  the  least  developed 
but  improved  over  time.  Approximately  70%  of  families  were 
functioning  in  the  normal  range  based  on  the  Family  Assessment 
Device  (FAD).  Pre-post  measures  of  family  functioning  revealed  less 
positive  family  functioning  over  time  (n=23).  It  may  be  important  to 
consider  sample  size  and  increased  strain  as  child  progresses  in  age.) 

3.  Child  development  appeared  to  improve.  (The  Denver  Developmental 
Screening  Test  II  (DDS-II)  was  used  to  monitor  healthy  growth  and 
development.  Results  based  on  the  testing  of  84  children  indicated  that 
80%  were  within  the  normal  range  for  development.  Of  the  22  children 
who  received  the  second  test  for  development,  91%  were  within  the 
normal  range  (an  increase  from  80%  at  baseline).  These  data  should 
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be  interpreted  with  caution,  however,  since  if  delays  are  going  to  occur 
or  worsen  they  are  more  likely  to  occur  in  the  2-3  year  old  age  group.) 

4.  Positive  impact  on  parent/child  interactions  (The  Adult-Adolescent 
Parenting  Inventory  II  (AAPI)  was  used  to  measure  parenting  skills 
according  to  five  key  attributes  (parental  expectations,  empathy  for  and 
awareness  of  the  child’s  needs,  definition  of  child  and  parental  roles, 
child’s  power  and  independence,  and  belief  in  corporal  punishment). 

Of  the  71  parents  who  completed  the  scale  initially,  most  scored 
average  or  high  on  the  first  three  attributes.  Of  those  who  responded  a 
second  time  (n=23),  there  were  fewer  parents  who  believed  in  corporal 
punishment  as  an  appropriate  form  of  discipline.  Four  mothers  have 
had  previously  apprehended  children  returned  to  their  care  as  a direct 
result  of  the  support  and  parenting  knowledge  gained  through  the 
Healthy  Families  program.) 

In  the  WestView  Healthy  Families  project  a broad  range  of  health  status 
indicators  was  also  used,  with  the  following  results  reported. 

■ The  most  frequently  cited  positive  techniques  included  stimulating 
motor  development,  stimulating  social/emotional/cognitive 
development,  and  encouraging  bonding/attachment. 

■ Family  Visitors  reported  that  60%  of  the  families  used  positive 
parenting  strategies  on  a regular  basis. 

■ 97%  of  children  tested  (using  the  Denver  Development  Screening 
Test  (DDST)  were  within  the  normal  range  for  growth  and 
development  including  cognitive,  motor,  social,  and  cultural  skills. 

■ On  average,  75%  of  parents  thought  they  had  been  able  to  make  a 
change  they  wanted  (such  as  looking  at  options  for  their  own  life, 
finding  ways  of  getting  financial  support).  Most  parents  agreed  that 
their  Family  Visitor  had  encouraged  them  to  figure  out  what  they 
wanted  to  do  or  how  they  should  do  it. 

■ The  majority  of  parents  reported  that  the  Program  had  a role  in 
boosting  their  confidence  in  managing  their  lives.  Also,  90%  of 
families  who  had  been  in  the  Program  for  three  months  had 
completed  a parent  action  plan. 

■ Overall,  86%  of  parents  were  able  to  name  at  least  one  community 
agency.  At  least  83%  of  all  parents  had  contacted  at  least  one 
other  agency  besides  the  Healthy  Families  Program.  Overall,  91% 
of  clients  thought  the  Program  had  helped  them  become  more 
confident  in  making  community  contacts. 
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Assumptions  about  Improved  Health  Status 

While  concrete  evidence  about  changed  health  status  (impact  on 
populations)  was  not  available  in  most  projects,  some  made  assumptions 
about  improved  health  status.  Anecdotal  evidence  from  the  Metis  project, 
for  example,  pointed  to  improved  health  of  specific  individuals  as  a direct 
result  of  the  placement  of  the  Settlement  Nurse  on  four  Metis  Settlements 
That  relation  will  need  to  be  tested  over  the  long  term. 


The  day  I knew  it  was  worth  it  was  the  day  we  were  having  a meeting  and  (name) 
(an  older  gentleman)  came  in  for  a flu  shot!  There  is  NO  WAY  he  would  have  had 
a flu  shot  if  the  nurse  hadn’t  been  on  the  Settlement! — Community  representative 
on  Steering  Committee  (Matthias,  2000). 


The  Elnora  project  reported  that  the  project  may  have  had  a potential 
impact  on  the  collective  psyche  of  the  community  in  terms  of  the 
engagement  in  the  initial  decision  making  about  the  project  and  the 
resultant  sense  of  achievement  and  ownership.  The  health  of  the 
community  might  be  further  impacted  by  the  decreased  level  of  anxiety 
about  the  availability  of  basic  health  services. 

Results  of  coalition  activity  in  the  Healthy  Okotoks  project  revealed 
assumptions  about  the  relation  between  opportunities  to  improve  health 
and  the  impact  on  their  community  population.  From  the  outset,  for 
example,  the  Coalition  regarded  primary  health  care  as 

a process  of  health  development  aimed  at  providing 
people  with  a level  of  health  that  permits  them  to  lead  a 
socially  and  economically  productive  life,  to  realize 
aspirations,  to  satisfy  needs,  and  change  and  cope  with 
the  environment. 

Creation  of  a community  garden,  involvement  in  an  annual  fall  clean  up 
and  safe  environments  for  children  to  play  in,  and  increased  awareness  of 
Fetal  Alcohol  Syndrome,  were  all  believed  to  contribute  to  improved  health 
status  of  the  community.  However,  the  connection  between  activities  and 
improved  health  status  will  be  very  difficult  to  objectively  determine. 

Similarly,  in  the  East  Central  project,  changes  in  the  perceived  health 
status  of  four  communities  were  noted  during  community  consultations. 
Most  notable  the  increased  attention  to  individual  and  self-responsibility 
for  health,  an  interest  in  preventing  health  problems,  and  a strong 
commitment  to  the  use  of  partnerships  to  maintain  community  health. 
Community  members  were  reported  to  have  a strong  awareness  of  the 
link  between  economic  development,  community  involvement  in  health 
issues,  health  planning,  and  health  status.  This  awareness  was  explicitly 
attributed  to  previous  experience  with  Healthy  Communities,  Physician 
Recruitment,  and  Seniors  Alliances  projects. 
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While  no  specific  determination  of  change  in  health  was  rendered  through 
the  Repositioning  project,  project  team  members  contended  that  the 
health  of  the  population  would  improve  by  conceptualizing  health  through 
the  lens  of  a “Total  Health  Process”  which  goes  beyond  a disease 
orientation  to  health  care.  The  project  team  also  suggested  that  the  model 
would  encourage  self-management  of  health  through  alignment  with  clear 
provider  accountability,  responsibility,  and  control. 

Summary  of  Learning 

The  majority  of  projects  identified  objectives  pertaining  to  impact  on 
populations,  but  a much  smaller  number  were  able  to  measure  that 
impact.  Challenges  included 

(a)  the  short  time  period  of  the  initiative, 

(b)  lack  of  baseline  data,  and 

(c)  lack  of  standardized  instruments  appropriate  to  measure  a broader 
determinants  of  health  perspective. 

However,  learning  about  impact  on  populations  (health  status)  was 
gleaned  from  several  projects. 

1 . Health  status  outcomes  provide  both  strategic  and  implementation 
markers  for  health  services.  However,  a broader  definition  of  health 
that  includes  a determinants  of  health  perspective  increases  the 
breadth  of  measures  which  can  be  used.  For  example,  economic 
(income,  employment)  and  social  (physical,  cognitive,  and  social 
development;  rates  of  family  violence)  indicators  should  be  considered 
in  determining  and  predicting  impact  on  populations.  Objectives 
concerning  impact  will  be  difficult  to  meet  without  baseline  information 
and  appropriate  measurement  tools. 

2.  Strategies  aimed  at  improving  the  health  of  populations  through 
integrating  services  (i.e.  physician/psychologist/psychiatrists;  rural 
health  services  teams;  nurses/physician/pharmacist)  show  promise, 
but  require  further  study. 

3.  Tracking  impact  in  high-risk  populations  is  very  difficult.  Some  clients, 
for  example,  do  not  have  health  care  numbers  or  home  addresses, 
and  often  use  only  first  names  or  aliases. 

4.  Building  rapport  between  the  client/patient  and  the  provider  has  been 
demonstrated  as  a critical  ingredient  in  building  client  strength  and 
confidence  to  engage  in  behaviours  that  improve  health  status. 
However,  some  groups  tend  to  be  dependent  on  direct  encouragement 
by  health  care  providers  for  the  maintenance  of  their  health.  A primary 
health  care  model  that  promotes  individuals  to  take  more  responsibility 
for  their  own  health  will  require  the  implementation  of  strategies  that 
are  effective  in  “weaning”  clients  away  from  this  dependency  yet 
provide  sufficient  support  to  facilitate  behaviour  change. 
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5.  Assumptions  around  what  interventions  will  impact  the  health  of 
populations  are  reasonably  easy  to  generate.  Proving  a causal 
connection  between  activities  and  improved  health  is  more  difficult. 
However,  community  development  measurements  and  assessment 
approaches  may  offer  a mechanism  to  link  aimed  at  improving  the 
health  status  of  individuals  and  positively  impacting  the  health  of  the 
population  overall. 
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Cost-Effectiveness 


Many  of  the  projects  that  made  up  the  Alberta  Primary  Health  Care  Project 
did  not  have  strong  approaches  to  assess  cost-effectiveness.  In  addition, 
many  of  the  evaluators  suited  to  evaluate  the  Alberta  projects  did  not  have 
substantive  backgrounds  in  economic  evaluation.  Three  approaches  were 
taken  to  address  these  issues.  Firstly,  a background  paper  was  developed 
introducing  the  concept  and  operationalization  of  cost-effectiveness 
(Howard,  Howard,  McCaffrey,  & Fassbender,  2000).  Secondly,  a 
framework  was  developed  to  assess  the  literature  related  to  cost- 
effectiveness  (Fassbender,  2000).  Thirdly,  a sustainability  project  was 
funded  to  better  understand  the  potential  for  human  services  initiatives  to 
generate  positive  long-term  outcomes  (Howard  and  Yates,  2000). 

Cost-effectiveness  assessment,  in  its  strictest  sense,  requires  measures 
of  cost  (dollars)  and  measures  of  effect  (psychological,  social,  behavioural, 
and/or  physical  construct).  In  addition,  a comparator  is  needed  to 
determine  if  one  approach  has  more  or  less  cost  and  more  or  less  effect, 
relative  to  previously  established  standards  of  costs  and  consequences. 


Figure  3.  Comparing  Cost  and  Effect  of  Programs 

Cost  Effectiveness  for  Alternate  Programs 
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There  are,  however,  several  forms  of  economic  evaluation  such  as  cost, 
cost-minimization,  cost-effectiveness,  cost-utility,  and  cost-benefit 
analysis.  A partial  economic  analysis  looks  at  costs.  A more  complete 
analysis  measures  both  costs  and  consequences. 
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Only  one  of  the  Alberta  projects13  was  able  to  meet  most  of  the  criteria 
required  for  a cost-effectiveness  analysis.  However,  several  projects  offer 
insight  into  cost-effectiveness  when  the  term  is  applied  to  a broader 
meaning  of  economic  analysis,  including  cost,  cost-minimization,  cost- 
effectiveness,  and  cost-benefit. 

The  Importance  of  Questions 

The  first  task  in  any  economic  evaluation  is  to  have  answerable  questions. 
The  Northeast  Community  Health  Centre  (NECHC)  provides  an  excellent 
example  of  this  exercise.  At  the  NECHC,  available  resources  and  short 
timelines  prohibited  the  completion  of  a formal  economic  evaluation. 
Guided  by  a health  economist,  the  management  team  and  internal 
evaluation  coordinator  developed  a series  of  potential  cost-effectiveness 
analysis  questions. 

Through  a series  of  discussions  the  NECHC  began  addressing  the  issue 
of  cost-effectiveness  by  framing  their  questions  around  a summative  and 
formative  framework  of 

a)  Is worth  doing?  (summative) 

b)  Is worth  doing  better?  (formative) 

From  this  exercise  the  project  was  able  to  generate  approximately  20 
potential  areas  of  investigation.  For  example:  Is  integrating  care  processes 
for  clients  who  are  new  immigrants  worth  doing?  Following  the 
development  of  their  potential  question  list  they  were  advised  to  assess 
each  question  according  to  pre-established  criteria14  to  ensure  that 
questions  were  asked  in  answerable  form  and  that  data  (outcomes  and 
costs)  were  anticipated  for  each  question.  A sample  of  those  criteria  are 
as  follows: 

1 . Do  the  questions  address  both  costs  and  effects  of  the  service(s)  or 
program(s)? 

2.  Do  the  questions  involve  a comparison  of  alternatives? 

3.  Are  all  the  important  and  relevant  consequences  for  each  alternative 
identified? 

4.  Are  consequences  to  be  measured  accurately  in  appropriate  physical 
units? 

5.  Are  all  important  and  relevant  costs  identified  for  each  alternative? 

6.  Are  costs  measured  accurately  in  appropriate  physical  units? 


13  Output  (services)  is  considered  the  measure  of  effectiveness  (more  output  may  not  be  more 
effective).  In  some  respects  the  8th  and  8th  study  referred  to  here  could  be  considered  a cost- 
minimization  study  rather  than  a cost-effectiveness  study. 

14  Drummond  (1997).  Methods  for  Economic  Evaluation  of  Health  Care  Programs.  Oxford  University 
Press. 
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Other  projects  posed  questions: 

• Which  time  shifts  in  the  Centre  (Community  Health  Centre)  are  the 
most  cost-effective? 

• What  are  the  activity  costs  of  the  Rural  Access  Program?  What 
effects  can  be  attributed  to  these  programs? 

• Is  the  Education  Program  worth  doing  relative  to  doing  nothing? 

• What  is  the  cost  of  Coalition  activity? 

• What  are  the  costs  in  introducing  an  Immunization  Improvement 
program?  Is  the  program  worth  doing? 


Developing  Appropriate  Data  Sets 

Answering  economic  evaluation  questions  require  the  collection  of 
appropriate,  accurate  and  sufficient  data.  The  majority  of  the  Alberta 
projects  were  unable  to  produce  data  on  which  to  conduct  a thorough 
economic  evaluation  for  one  or  more  of  the  following  reasons: 

• the  short  duration  of  the  project  precluded  sufficient  data  collection, 

• data  requirements  were  not  sufficiently  detailed  early  enough  in  the 
project, 

• data  was  not  available  for  project  use,  and 

• data  sets  were  incomplete  (i.e.,  data  were  captured  for  some 
variables  but  not  others). 

In  addressing  cost-effectiveness,  projects  tended  to  either  point  toward  the 
potential  for  the  project  to  be  cost-effective,  or  they  made  a direct  attempt 
at  collecting  data  to  support  current  or  future  assessments  of  economic 
value. 

A select  number  of  Alberta  Primary  Health  Care  projects  tackled  cost- 
effectiveness  directly  and  produced  data  sets  useful  to  conduct  a full  or 
partial  economical  evaluation.  For  example,  at  the  8th  and  8th  Health 
Centre,  an  economist15  undertook  to  develop  and  assess  the  robustness 
of  a cost-effective  methodology  suitable  for  comparing  operations  across 
work  shifts.  The  ultimate  objective  was  to  apply  the  methodology  in 
comparing  the  performance  of  the  medical  care  component  of  the  Centre 
with  that  of  other  settings.  The  cost-effectiveness  study  suggested  that,  in 
the  early  stages  of  Centre  operations,  the  night  shift  was  less  cost- 
effective  than  the  other  two  shifts.  As  waiting  times  increased  during  the 
other  shifts,  however,  and  partly  due  to  differential  trends  in  costs  and 
patient  loads,  the  night  shift  was  estimated  to  become  relatively  more  cost- 


15  Auld,  2000. 
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effective.  The  econometric  methodology  was  judged  to  be  feasible  for 
inter-provider,16  inter-facility  comparisons. 

The  8th  and  8th  Health  Centre  project  illustrates  both  the  data  requirements 
and  methodology  that  may  be  required  in  examining  costs  and  effect 
relationships  in  primary  health  care.  Data  consisted  of  the  complete, 
disaggregated  patient  billing  records  for  the  Clinic  from  March  1997 
through  August  1999.  Additional  data  sets  were  compiled  for  variable 
costs  by  hour  for  a selected  set  of  days.  A data  summary  is  presented  in 
Table  3. 


Table  3.  Operationalization  of  Variables 


Variable 

Operationalization 

Output  measure 

Services  provided 

Wait  times 

Wait  time  of  patients — $10.00  per  hour 

Shift  time 

Morning,  day,  night 

Patient  load 

Number  of  patients  seen  per  shift 

Time 

Day  and  month  of  service 

Physician 

Physician  providing  the  service 

Shift  cost 

Personnel  costs 

Sources:  Auld,  2000 


A regression  model  was  used  to  differentiate  the  complexity  of  time- 
dependent  influences  on  cost-effectiveness.  It  is  important  to  note  that 
wait  times  were  considered  an  important  variable  in  this  economic 
evaluation.  Ignoring  wait  times  when  computing  cost-effectiveness  is 
problematic,  since  wait  times  are  considered  an  important  contributor  to 
quality  of  care. 

While  cost-effectiveness  as  an  economic  evaluation  procedure  requires  a 
comparator,  this  is  not  true  of  all  economic  evaluations.  All  economic 
analyses  do,  however,  depend  on  the  accurate  collection  of  data  on  costs 
and  impact  (consequences).  The  CPPOA  model,  used  in  the  sustainability 
studies,  is  an  example  of  how  formative,  activity-based  cost  and  effect 
analysis  can  be  conducted  at  the  program  level  by  practitioners  (see 
Figure  6).  The  model  introduces  a structure  for  data  collection  called  a 
Resources/Procedures  Matrix  (see  Table  4 for  an  example). 


16  Comparator  providers  would  have  to  be  implementing  variable  shift  time  operations. 
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Table  4.  Sample  Matix  (Health  Authority  5 Clinic  Data) 


Procedures  - Observable  Activities 

PHYSICIAN 

CLINIC 

CREMONA 

Clinical  Services 

Administration 

Resources 

Data  Source 

Time 

Clinic  Invoices 

4 hours/wk 
$250.00  stipend/wk 

4.5  hrs/wk  @ 
$12.00/hr=$54.00/wk 

Travel 

Clinic  Invoices 

1 hour/wk 

88  km  @ .28  = $24.64/wk 

nil 

Space 

Rent  Invoice 

$300. 00/month 
$75.00/wk 
Cleaning  $10.00/wk 

nil 

Materials 

Project  Budget 

Phone  = $55. 00/month 
$13.75/wk 
Pharmacy  Supplies 
$106.49  set  up  cost 
$5.00/wk  maintenance 
Clinic  Supplies 
$950.00  set  up  cost 
$10.00/wk  maintenance 
Office  Furniture 
$400.00  set  up 
Donated  exam  table 

Fax  Machine  @ 
$360.00 

Physician  Clinic  Cremona  - Pod  2 

Total  cost  per  week:  $442.39 

Average  client  per  week:  14 

Average  cost  per  client:  $29.50 

Source:  Frere,  2000;  Howard  and  Yates,  2000 


In  a cost-effectiveness  analysis,  a further  connection  can  be  made 
between  resources,  procedures,  processes,  and  outcomes. 

Figure  4.  Self-reported  Hospital  Utilization 


In  the  case  of  the  Health  Authority  5 Rural  Health  project,  data  were 
collected  on  costs  and  community  measures  of  self-perceived  health 
status,  attitudes  toward  health,  and  health  behaviour.  Data  from  the  rural 
communities  receiving  health  services  were  compared  to  a similar 
community  not  receiving  health  services.  These  data  were  then  used  by 
the  Region  to  determine  the  extent  to  which  they  felt  the  program  was 
worth  continuing  (see  section  on  “worth”). 
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Potential 


Several  projects  suggested  that  processes  and  procedures  being 
implemented  in  the  projects  may  have  the  potential  to  contribute  to  more 
cost-effective  health  service  delivery.  For  example,  in  the  Calgary 
Repositioning  project,  the  combination  of  the  mapping  tool,  game,  and 
population  modeling,  in  theory,  provide  the  means  to  match  population 
needs  to  resource  allocation.  The  project  provided  some  data  around  the 
applications  that  suggest  the  potential  for  cost-effectiveness  through 
reconfigured  services.  A cost  saving  of  7.9%  was  predicted  for  current 
versus  reconfigured  service  delivery  for  the  amyotrophic  lateral  sclerosis 
(ALS)  population.  Furthermore,  the  project  team  suggested  that  between 
10%  and  30%  of  resources  could  be  released  at  the  same  time.  The 
independent  evaluators  were  unable  to  verify  these  claims,  however. 

The  evaluation  of  the  Alexandra  Community  Health  Centre  (CHC)  tried  to 
address  cost-effectiveness  through  a study  on  shadow  billing.  The  internal 
evaluators  noted  that  this  section  of  the  evaluation  was  problematic  due  to 
both  over-  and  under-reporting  of  services  provided  by  physicians. 
Although  the  evaluation  did  not  include  direct  measures  of  cost- 
effectiveness,  the  following  indicators  of  cost-effectiveness  are  provided  in 
the  model  of  care. 

1 . The  Alexandra  CHC  is  multi-disciplinary  which  should  allow  for 
comprehensive  care  by  the  most  appropriate  provider. 

2.  Physicians  are  contracted  as  opposed  to  paid  under  a to  fee-for- 
service  schedule,  which  generally  indicates  a willingness  to  partner 
with  other  health  care  providers  and  provide  services  by  telephone  as 
appropriate.  By  managing  patients  through  the  CHC  model,  the 
physicians  are  able  to  manage  more  difficult  cases  with  less  referral  to 
specialists  or  hospitals,  particularly  for  psychiatric  cases. 

In  the  CUPS  project,  use  of  a collaborative  practice  model  contributed 
indirectly  to  cost-effective  service  delivery.  The  nurse  practitioner  was  able 
to  monitor  chronic  and  complex  cases  between  doctors’  visits.  In  addition, 
there  is  increased  opportunity  for  communication  with  providers,  other 
health  care  organizations,  detox  centres,  and  police  services. 

Although  no  evidence  exists  to  make  a determination  of  the  cost- 
effectiveness  of  telehealth  in  the  three  northern  communities,  one  would 
expect  that  cost  savings  would  result  from  residents  not  having  to  leave 
their  home  communities  or  health  providers  not  being  required  to  travel 
into  remote  communities. 

The  Collectives  project  was  not  a cost-effectiveness  study.  Nevertheless, 
there  are  some  results  in  this  project  that  suggest  that  the  intervention 
should  be  further  refined  with  a controlled  study,  followed  by  a formal 
economic  evaluation. 
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1 . Health  status  of  the  unwell  group  was  maintained. 

2.  Use  of  health  services  (i.e.,  self-reported  physician  and  Emergency 
Room  visits)  declined. 

3.  Improved  compliance  over  the  long-term  is  believed  to  be  cost  saving. 

4.  Avoidance  of  drug  reactions  and  drug  interactions  and  coordinating 
non-drug  related  services  assists  patients  to  better  manage  their  own 
health. 

The  Alberta  College  of  Physicians  and  Surgeons  and  Department  of 
Family  Medicine  in  the  Faculty  of  Medicine  and  Dentistry  agree  in  principle 
about  enrollment  of  patients  in  collaborative  practice  by  members  other 
than  physicians  (this  has  the  potential  to  increase  cost-effectiveness  of  the 
collaborative  practice  model). 

According  to  the  evaluators,  the  annual  cost  of  the  WestView  Healthy 
Families  program  is  approximately  $295,000.  As  of  March  31,  2000,  there 
were  45  families  enrolled  in  the  program.  This  works  out  to  approximately 
$6,500  per  family,  which  is  comparable  to  similar  programs  delivered 
under  Healthy  Families  America  in  the  United  States.  The  rationale  for 
programs  of  this  nature  is  their  potential  to  reduce  costs  of  social 
underdevelopment  and  violence.  Healthy  Families  (Alberta)  should  lend 
itself  to  a cost-benefit  economic  evaluation,  providing  the  evaluation  is 
planned  early  enough  to  establish  mechanisms  for  the  collection  of 
appropriate  data  sets. 

Establishing  Baseline 

Those  projects  making  some  attempt  to  establish  costs,  effects,  or  both 
were  able  to  provide  effectiveness  data,  cost  data,  or  both  kinds  of  data  to 
be  used  for  comparison  with  future  iterations  of  the  same  program  or  a 
comparable  program.  They  were  able  to  establish  some  baseline  on  which 
to  develop  economic  questions. 

Since  all  projects  were  accountable  for  funds  received,  the  Alberta 
projects  each  established  baseline  data  on  costs  of  start-up  and  project 
implementation  of  the  approaches  used.  Some  projects  conducted  activity- 
based  assessments  and  have  a reasonable  breakdown  of  the  costs  per 
activity  engaged.  Others  have  at  least  the  broader  project  costs.  In  a few 
instances,  projects  such  as  8th  and  8th,  HA5  Rural  Health,  Misericordia 
Health  Lifestyle,  Okotoks  Community  Coalition,  and  Keweetinok 
Immunization  projects  measured  effects  and  established  baselines  on  the 
costs  and  effects. 

The  Healthy  Okotoks,  Misericordia,  and  HA5  Rural  Health  programs  were 
included  in  a sustainability  study  conducted  within  the  Alberta  umbrella 
project.  Detailed  costing  of  the  projects  is  outlined  in  that  study.  The 
sustainability  study  provides  effectiveness  and  cost  data  that  can  be  used 
for  comparison  with  reiterations  of  these  projects  and  with  different 
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approaches.  The  current  data  would  assist  an  economic  evaluator  in 
determining  the  “worth”  of  these  approaches  relative  to  the  approach  that 
was  in  place  before  the  study  began  or  another  approach  altogether. 
Keewetinok  established  data  to  assist  in  determining  the  effort  required  to 
significantly  improve  the  immunization  rates  in  their  Regional  Health 
Authority. 

The  following  is  a list  of  projects  that  offer  more  than  aggregate  costs  of 
the  project. 


Table  5.  Projects  With  Measures  of  Effect 


Activity-Based  Cost  Analysis 

Measures  of  Effect 

8th  and  8th  CHC 

Services  provided 

Health  Authority  5 Rural  Health 

Self-perceived  community  health 
status,  attitude,  and  behaviour 

Healthy  Okotoks  Coalition 

Decision-making  resource  utilization 

Misericodia  Health  Lifestyle 
Education  Centre 

Cholesterol 

Keeweetinok  Immunization 

Immunization  rates 

Assessment  of  Worth 

Assessing  worth  or  value  is  not  always  obvious.  While  resource 
consumption  may  produce  results,  ultimately  some  authority  has  to  decide 
whether  the  consequences  of  expending  the  resources  are  worthwhile. 
That  decision  is  often  made  within  a context  broader  than  a program  or 
project.  The  question  often  becomes  one  of  allocation  of  resources  to 
several  projects  and  programs.  Therefore,  an  initiative  is  often  measured 
against  the  merits  of  other  initiatives  competing  for  limited  resources. 

An  examination  of  the  cost-effectiveness  data  from  the  HA5  project 
suggest  that  the  gains  made  in  each  of  the  test  communities  may  be  worth 
achieving.  However,  the  region  must  determine  whether  the  results 
suggest  the  program  is  worth  doing  again. 


Table  6.  Cost  Per  Client  Across  Four  Clinics 


Youngstown 

Cessford 

Cremona 

Hussar 

Cost/Week 

$295.00 

$323.00 

$442.39 

$390.85 

Client/Week 

2 

5 

14 

8 

Cost/Client 

$147.50 

$64.69 

$29.50 

$51.60 

Source:  Frere,  2000 


In  the  Keeweetinok  Immunization  project  the  question  arises:  Was  the 
52%  increase  in  immunizations  (approximately  20%  increase  in 
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immunization  rates  in  the  region)  worth  the  47. 517  days  allocated  to 
contact  and  interview? 

Immunization  rates  for  1997  in  relation  to  their  deviation  from  the 
Canadian  National  Goals  and  Targets  as  set  out  in  the  Canadian  National 
Report  on  Immunization,  1996. 


Table  7.  Keeweetinok  Lakes  Regional  Health  Authority  Regional 


Disease* 

Average  NACI 
Goals  (%) 

Regional 
Statistics  (%) 

Regional  Deviation 
from  Goals  (%) 

DPT 

96 

62 

34 

Hib 

97 

61 

36 

MMR 

97 

86 

11 

Poliomyelitis 

97 

62 

35 

^Immunizations  that  are  given  simultaneously,  and  therefore  have  identical  uptake  rates 
are  grouped  together. 

Source:  Van  Herk-Auger,  Denton,  Beyon,  and  Brockway,  2000 


Table  8.  Immunization  Statistics  for  1998  and  1999 


Region 

UTD 

UND 

Imm.  Rate  for 
Eligible  Pop. 

1998  Imm.  Rate  for  1999  Imm.  Rate 
Region  for  Pop. 

TOTAL 

605 

309 

66.2% 

66.5%  86.8% 

Note:  The  sample  population  decreased  slightly  to  a total  of  908  from  914  and  the 
immunization  rate  in  the  sample  population  became  65.6%  by  the  time  the  interviewing 
process  began. 

Source:  Van  Herk-Auger,  Denton,  Beyon,  and  Brockway,  2000 


The  Healthy  Okotoks  Coalition  was  successful  in  achieving  24  community 
changes.  In  the  experience  of  the  Coalition,  a great  deal  of  time  and  effort 
goes  into  planning  products  and  community  actions  before  a community 
change  takes  place.  From  a review  of  meeting  minutes  and  project 
records,  it  appears  that  it  took  one  and  one-half  years,  31  planning 
products  and  15  community  actions  to  achieve  one  community  change. 
Data  were  plotted  on  charts  such  as  in  Figure  5 taken  from  the  evaluator’s 
report. 


17  Development  costs  do  not  have  to  be  considered  in  this  scenario,  since  outside  funding  provided  a 
reusable  mechanism  for  improving  immunizations.  The  region  can  replicate  this  study  without 
additional  development  costs. 
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Figure  5.  Cumulative  Planning  Products  for  HOC  Total  Projects 


Source:  Hodson,  Anderson,  Anderson,  Raine-Travers,  and  Carey,  2000 

The  Misericordia  project  revealed  “no  significant  difference”  in  cholesterol 
measures  between  the  treatment  group  and  the  control  group.  Decision- 
makers at  Misericordia  will  have  to  determine  whether  the  data  collected 
from  the  qualitative  study  merit  continuation  of  the  program.  Using  the 
CPPOA  Matrix,  for  example,  activity-based  costing  was  broken  down  into 
units  as  small  as  start-up  and  ethics  review.18 

Table  9.  START  UP  and  ETHICS 


Procedures  - Observable  Activities 

START  UP 

Proposal  Development 

Resources 

Data  Source 

Administration 

Coordinator 

estimates 

Document  review 
2 hr.  @ 135.00 

Coordinator/ 

Physician 

Coordinator 

estimates 

Consultation  with  principal  investigators 
Proposal  preparation 
18  hr.  @ 135.00 

Psychologist 

Project  Records 

Consultation  with  coordinator 
2 hr.  @ 100.00 

Materials 

Coordinator 

estimates 

Word  processing  materials 
Minimal:  25.00 

Total  Start  Up:  $2950.00 

Procedures  - Observable  Activities 

ETHICS 

Ethics  Proposal  Development 

Resources 

Data  Source 

Administration 

Coordinator 

estimates 

Document  review 
1 hr.  @ 135.00 

Coordinator/ 

Physician 

Coordinator 

estimates 

Consultation  with  principal  investigators 
Consultation  with  methodologist 
Communication  with  Ethics  Board 
Ethics  proposal  preparation 
5 hr.  @ 135.00 

Psychologist 

Project  Records 

Consultation  with  coordinator 
2 hr  @ 100.00 

Research 

Methodologist 

Methodologist 

Consultation  / power  calculations 
1 hr  @ 135.00 

Materials 

Coordinator 

estimates 

Word  processing  materials 
Minimal:  25.00 

Total  Ethics:  $1170.00 

18  Start-up  and  ethics  are  typically  protocol  driven  costs  that  are  not  included  in  an  economic 
evaluation.  These  costs  are  incurred  for  the  purposes  of  the  study  and  not  to  provide  services. 
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Developing  Economic  Evaluation  Capacity 

Several  projects  were  able  to  contribute  to  regional,  organizational,  and 
personal  capacity  to  address  issues  of  cost-effectiveness.  Some  of  the 
more  prominent  examples  of  capacity  building  to  do  economic  evaluations 
occurred  in  the  Northeast  CHC  and  the  three  sustainability  projects:  HA5 
Rural  Health,  Okotoks  Community  Coalition,  and  the  Misericordia  Health 
Lifestyle  Education  Centre.  The  8th  and  8th  project  was  independently 
contracted  and  the  resulting  report  was  written  in  an  “academic/technical” 
style  which  makes  it  difficult  for  those  without  cost-effectiveness  training  to 
follow.  A “plain  language”  revision  of  this  report  would  improve  its  potential 
for  more  general  usage. 

The  three  sustainability  projects  engaged  in  a formative  activity  based 
cost-effectiveness  analysis  using  a proven  human  services  economic 
evaluation  model.19  Staff  implementing  these  projects  were  provided  with 
inservices  on  the  model. 


Figure  6.  Basic  CPPOA  Model 


Source:  Measuring  and  Improving  Cost,  Cost-Effectiveness,  and  Cost-Benefit  for  Substance  Abuse 

Treatment  (Yates,  1999).  Model  reproduced  with  permission. 


No  substantive  background  in  accounting  or  research  is  needed  to  use  the 
methods  outlined  for  use  of  the  CPPOA  model  (see  Figure  6).20  Some 
assistance  may  be  required  in  initial  development  of  parameters 
(operationalization  of  variables  and  measurement)  and  in  assessing  the 
statistical  relationship  of  variables.  The  model  has  been  well  researched 
and  tested  within  substance  abuse  treatment  programs. 

The  CPPOA  model  was  developed  based  on  observations  of  how  staff 
and  experienced  program  decision-makers  viewed  program  change  during 
implementation,  what  they  learned,  and  how  they  applied  their  findings  to 
improve  the  program.  The  model  was  designed  to  help  staff  and 
administrators  of  human  services  programs  provide  concrete  evidence  of 
their  contribution  to  stakeholders  through  cost-effectiveness  and  cost- 
benefit  analysis.  A sample  of  steps  are  provided  in  Table  10. 


19  See  Measuring  and  Improving  Cost,  Cost-Effectiveness,  and  Cost-Benefit  for  Substance  Abuse 
Treatment  (Yates,  1999). 

20  Strictly  speaking,  no  training  is  required  to  perform  an  economic  evaluation — difficulties  may  arise  in 

the  interpretation  of  technical  language  and  in  choosing  measurement  methods. 
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Table  10.  Sample  Timetable  for  Cost,  Cost-Effectiveness,  and  Cost 
Benefit  Analysis 


STEPS 

WEEKS  1 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

1.  Identify  key 
players,  interest 
groups  and  a 
coordinator 

► 

2.  Assign 

responsibilities  for 
each  step 

3.  Tailor  this 

timetable  to  your 
program 

-* 

► 

4.  Develop  or  refine 
a reporting  plan 

-► 

5.  Describe  program 
components  and 
desired  outcomes 

6.  Choose  and  test 
cost  measures 

7.  Develop  and  test 
effectiveness 
measures 

-► 

8.  Choose  and  test 
benefit  measures 

9.  Implement  regular 
collection  ana 
reporting  of  cost 
measures 

w 

10.  Regularly  collect 
and  report 
effectiveness 
measures 

w 

1 1 . Regularly  collect 
i and  report  benefit 

measures 

w 

12.  Perform  first  cost 
analysis 

► 

13.  Perform  first 
effectiveness 
analysis 

— ► 

14.  Perform  first 
benefit  analysis 

— ► 

15.  Perform  first  cost- 
effectiveness 
analysis 

— ► 

16.  Perform  first  cost- 
benefit  analysis 

— ► 

17.  Perform  monthly 
analysis  of  cost, 
cost-effectiveness, 
and  cost-benefit 

-► 

Source:  Measuring  and  Improving  Cost,  Cost-Effectiveness,  and  Cost-Benefit  for  Substance  Abuse 


Treatment  (Yates,  1999). 


Implications  in  Conducting  Economic  Evaluations 


While  evaluation  of  cost-effectiveness  of  Alberta  Primary  Health  Care 
projects  was  challenging,  some  of  the  challenge  was  associated  with 
project  and  evaluator  assumptions  that  cost-effectiveness  is  a technically 
intensive  exercise.  Although  summative  cost-effectiveness  can  require 
certain  expertise  and  resource  commitments,  formative  cost-effectiveness 
is  often  routinely  conducted  on  an  informal  basis  by  administrators  and 
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managers — for  example  by  administrators  at  hospitals  and  regional  health 
authorities  when  making  drug  formulary  decisions. 

However,  as  evaluators  of  the  NECHC  concluded,  the  primary  health  care 
service  delivery  model  is  distinctly  different  from  the  traditional  acute  care 
model,  upon  which  much  of  the  economic  evaluation  literature  is  based. 
For  example,  much  of  the  economic  evaluation  literature  has  focused  on 
the  efficiency  of  products.  Process  and  organizational  innovations,  such  as 
those  occurring  at  the  NECHC,  are  more  difficult  to  describe  and  therefore 
more  difficult  to  evaluate. 


However,  “the  pursuit  of  efficient  practice  is  not  merely  about  reducing  costs.  '21 
Sustainability  of  primary  health  care  reform  requires  that  decisions  be  made  on 
the  basis  of  evidence  in  accordance  with  the  stated  objective  of  health  care 
reform — better  health  at  lower  cost — or  cost-effectiveness  (in  Howard,  Howard, 
McCaffrey,  and  Fassbender,  2000). 


As  independent  evaluators,  it  was  important  to  distinguish  between 
evaluating  the  process  of  establishing  whether  services  are  efficient  and 
evaluating  the  outcomes  associated  with  such  a process.  A scientific  or 
peer-review  process  may  be  a more  appropriate  vehicle  to  evaluate  the 
outcomes  of  an  economic  evaluation.  More  importantly,  it  was  not 
reasonable  to  expect  a formal  economic  evaluation  to  be  performed  within 
the  short  time  period  of  an  evaluation,  such  as  the  NECHC. 

Collecting  cost-effectiveness  evidence  is  a resource-intensive  exercise. 
Moreover,  concern  was  expressed  that  conclusions  regarding  the 
performance  of  services  could  be  made  on  the  basis  of  preliminary 
evidence.  This  illustrates  the  importance  of  considering  the  decision- 
making context  of  a cost-effectiveness  study. 

Sustainability 


As  indicated  in  the  introduction,  a sustainability  project  was  undertaken  to 
better  understand  the  potential  for  human  service  initiatives  to  generate 
positive  long-term  outcomes.  The  goals  of  the  sustainability  project  were 
to  provide  the  Alberta  Primary  Health  Care  Project  with  the  opportunity  to 

(a)  understand  the  concept  of  sustainability  and  its  relationship  to 
advancing  primary  health  care,  and 

(b)  provide  an  approach  and  tools  to  assess  the  potential  for 
sustaining  outcomes. 


A definition  and  operationalization  of  sustainability  was  not  driven  by  a 
desire  to  determine  the  potential  for  extension  of  any  of  the  27  projects.22 
While  units  of  investigation  were  individual  projects,  the  approach  to 


21  Maynard  A.  Logic  in  Medicine:  An  economic  perspective.  BMJ  1987;  295:1537-1541. 

22  In  fact,  each  project  was  made  aware,  early  in  the  approval  process,  that  funding  through  the  Health 

Transition  Fund  would  be  time  specific — two  years 
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sustainability  was  to  identify  elements  and  characteristics  of  initiatives23 
that  might  support  achieving  outcomes  over  the  long  term,  without 
extending  long-term  external  support.  In  addition,  a proactive  approach  to 
assessing  the  potential  for  sustainability  of  human  services  initiatives  was 
modeled. 

To  assess  potential  for  sustainability  following  the  activity-based  costing 
exercise,  projects  selected  for  the  sustainability  study  were  asked  to 
respond  to  a sustainability  tool  to  assess  their  potential  for  sustainability. 
The  tool  was  based  on  an  assessment  of  dependency  (Self- 
determination/Self-reliance— SDR  model)  through  evaluating 
organizational  and  community  development  (Howard  and  Howard,  2000). 

Scales  were  developed  for  the  dimensions  of  (a)  accountability  and 
responsibility,  (b)  decision  making  authority,  (c)  gathering  and  sharing 
information,  (d)  knowledge  and  skills,  and  (e)  resource  mobilization. 
Members  of  the  project  team  were  then  asked  to  rate  each  dimension.  A 
sample  of  the  exercise  for  one  of  the  dimensions  is  given  in  Table  1 1 . 


Table  11.  ITEMS  C.  Information  Gathering  and  Sharing 


Program  is  aware  of  issues  and  actions  impacting  the  program. 

1 | 2 | 3 | 4 | 5 

[ 6 | 7 | 8 

1 9 

1 10 

LOW 

HIGH 

Program  shares  a common  understanding  of  issues  and  actions  impacting  the  program. 

1 | 2 | 3 | 4 | 5 

6 | 7 | 8 

! 9 

1 io 

LOW 

HIGH 

Gaps  in  important  information  are  identified. 

1 | 2 | 3 | 4 | 5 

6 | 7 | 8 

! io 

LOW 

HIGH 

Diverse  sources  of  information  are  sought. 

1 | 2 | 3 | 4 | 5 | 

1 6 | 7 | 8 

1 9 

1 io 

LOW 

HIGH 

Information  is  gathered  from  all  stakeholder  groups  impacted  by  the  program. 

1 1 2 | 3 | 4 | 5 | 

6 | 7 | 8 

! 9 

! io 

LOW  HIGH  1 


Source:  Howard  and  Howard,  2000 

In  the  end,  each  assessment  resulted  in  a visual  indication  that  helped  the 
project  administration  determine  weaknesses  and  strengths  for  potential 
sustainability  of  outcomes.  A total  low  score  on  any  one  dimension  was 
considered  a weakness  in  potential  to  sustain  the  project’s  outcomes  over 
a longer  period  of  time.  A graphic  representation  of  this  exercise  is 
provided  in  Figure  7. 


23  Initiative  is  a broad  term  that  includes  human  services  projects  and  programs. 
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Figure  7.  Sustainability  Grid 


□ 

Accountability/Responsibility 

, 

□ 

Decision  Making  Authority 

^ — 1 ► 

□ 

Gathering/Sharing  Information 

Knowledge/Skill 

« 

□ 

Resource  Mobilization 

« 1 * 

Source:  Howard  and  Howard,  2000 


The  SDR  Model  identifies  the  key  elements  believed  by  the  principal 
investigators  to  plot  the  extent  to  which  an  initiative  has  the  will  to 
determine  its  own  destiny.  They  are 

a)  commitment  to  and  a sense  of  responsibility  to  stakeholders, 

b)  authority  to  make  critical  decisions, 

c)  information  that  is  relevant  and  timely  to  make  appropriate 
decisions, 

d)  knowledge  and  skill  to  reflect  on  and  respond  to  information  and 
decisions  made,  and 

e)  resources  available  to  participants  (whether  they  exist  in  the 
external  or  internal  environment). 

Through  an  increased  sense  of  SDR,  initiatives  move  forward  with 
confidence  and  a sense  that  they  can  survive  without  extensive  external 
support,  while  continuing  to  acknowledge  that  they  exist  within  an 
interrelationship  with  their  external  environment.  Most  importantly, 
initiatives  willingly  and  actively  engage  in  an  interrelationship  with  their 
external  environments  to  fulfill  their  needs  and  exploit  opportunities.  This 
engagement  is  a movement  toward  sustainability. 
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Summary  of  Learning 

While  each  project  must  be  examined  in  its  uniqueness,  perspectives  on 

cost-effectiveness  emerged  from  the  aggregate  experiences  of  the  Alberta 

Primary  Health  Care  project  as  a whole. 

1.  A distinction  should  be  made  between  cost-effectiveness  as  economic 
evaluation  and  cost-effectiveness  as  one  approach  for  doing  economic 
evaluations.  Cost-effectiveness  as  an  approach  requires 
measurements  of  cost  and  effect  and  the  presence  of  a comparator. 

2.  Planning  is  essential  for  analyses  of  cost,  cost-effectiveness,  and  cost- 
benefit.  The  planning  process  requires  the  articulation  of  an 
answerable  question,  the  development  of  appropriate  data  sets,  and 
some  assessment  of  worth. 

3.  While  a perception  of  complexity  and  difficulty  of  doing  cost- 
effectiveness  may  prevail,  basic,  plain  language  in-servicing  on 
economic  evaluation  can  prepare  staff  and  administrators  to  perform 
some  level  of  economic  analysis. 

4.  Collection  of  cost  data  leads  to  concern  about  inappropriate  decisions 
if  health  consequences  (outcomes)  are  not  taken  into  account. 

5.  Data  collection  strategies  need  not  be  overly  intrusive.  Often  a time- 
series  approach  to  data  collection  is  more  economical  and  definitely 
less  intrusive  on  the  time  of  participants.  Often,  data  collected  at 
predetermined  intervals  can  be  used  to  extrapolate  to  the  entire  project 
or  program. 

6.  Economic  evaluations  are  not  necessarily  summative.  Often,  economic 
information  is  valuable  during  implementation  and  operation.  Decisions 
to  adjust  or  change  direction  can  be  determined  by  interim  cost  and 
effect  data. 

7.  Costs,  effects,  and  benefits  must  be  considered  in  advancing  primary 
health  care  to  improve  health  and  to  enhance  sustainability  of  health 
systems.  However,  there  are  implications  to  conducting  economic 
evaluations  in  primary  health  care  settings.  For  example,  much  of  the 
economic  evaluation  literature  has  focused  on  the  efficiency  of 
products.  Process  and  organizational  innovations  are  more  difficult  to 
describe  and  therefore  more  difficult  to  evaluate.  Some  of  the 
constructs  in  these  environments  are  difficult  to  operationalize  (e.g., 
wait  times,  client  satisfaction,  service). 

8.  While  cost  is  an  important  element  in  determining  the  sustainability  of 
long-term  outcomes,  other  environmental  factors  need  to  be 
considered  in  sustainability  assessment.  An  initiative  can  benefit  from 
assessing  its  relationship  with  its  environment,  both  internal  and 
external.  Often  this  contextual  assessment  can  illustrate/identify  why 
some  programs  produce  long-term  sustainable  outcomes  for 
reasonable  cost,  while  others  appear  to  provide  little  long-term  benefit 
at  excessive  cost. 
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Transferability 


In  order  to  demonstrate  transferability  or  generalizability  it  is  essential  to 
consider 

(a)  the  similarity  between  the  contexts  in  which  the  study  was 
conducted  and  where  it  will  be  applied  (i.e.,  the  extent  to  which  the 
study  can  be  replicated),  and 

(b)  trustworthiness  of  the  results  (i.e.,  the  extent  to  which  rigorous 
processes  were  used  to  design,  implement  and  evaluate  the  study, 
so  that  findings  can  be  believed). 

Mindful  of  these  two  key  factors  for  potential  transfer,  several  projects  offer 
learning  and  tools  that  may  be  useful  to  others. 

The  Importance  of  Context 

Understanding  the  context  in  which  a project  was  developed  and 
implemented  is  very  important  to  determining  the  extent  to  which  the 
process  can  be  replicated  elsewhere.  The  Northeast  Community  Health 
Centre  project  has  prepared  a document  which  captures  the  history  and 
process  of  its  implementation  (“Collaboration  in  Action:  The  Story  of 
Edmonton’s  Northeast  Community  Health  Centre"). 


The  tornado  that  struck  Edmonton  on  July  31,  1987,  crystallized  the  resolve  of  the 
northeast  communities  to  secure  improved  access  to  emergency  services.  (Burke, 
2000). 


Some  of  the  essential  elements  of  that  process  include  development  of  a 
common  vision,  support  for  the  service  delivery  model,  participatory 
leadership,  and  formative  evaluation  activities.  Co-location  of  public  health 
and  relevant  primary  health  care  services  was  also  an  important  pre- 
requisite for  health  professional  collaboration  and  integrated  service 
delivery  at  the  Northeast.  Key  learning  of  the  project  team  suggests  that 
having  an  evaluation  in  place  at  the  point  of  project  initiation  and  using 
formative  evaluation  findings  to  inform  action  were  instrumental  to 
successfully  operationalizing  this  community  health  centre. 

Tools,  Templates,  and  Models 

A major  legacy  of  the  Alberta  Primary  Health  Care  project  is  the  variety  of 
tools,  templates  and  models  developed  and  used.  These  instruments  and 
models  are  useful  to 
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(a)  gather  client  information; 

(b)  map  health  services; 

(c)  build  consensus  and  capacity  across  provider  groups,  patients; 
caregivers,  families,  and  community  volunteers; 

(d)  evaluate  an  information  technology  system; 

(e)  assess  immunization  compliance; 

(f)  assess  children’s  mental  health  status; 

(g)  establish  use  and  attitudes  towards  accessibility  of  an  Emergency 
Department; 

(h)  gather  perspectives  of  health  providers; 

(i)  assess  needs  of  patients  and  caregivers; 

(j)  evaluate  quality  of  medication  use; 

(k)  establish  processes  for  team  building; 

(l)  facilitate  communication  among  health  professionals; 

(m)  conduct  economic  evaluations;  and 

(n)  assess  sustainability. 

Tools 


Client  Information 

A Health  Determinants  Instrument  was  used  in  the  evaluation  of  the 
Alexandra  CHC  that  has  potential  to  provide  useful  information  about 
clients  at  Alexandra  on  a longitudinal  basis.  The  tool  may  also  have 
potential  use  in  other  primary  health  care  settings  to  provide  comparative 
data. 

Codes  for  Shadow  Billing 

During  the  shadow  billing  process  used  at  Alexandra,  a number  of  codes 
were  developed  to  account  for  services  not  considered  billable  in  the 
current  fee-for-service  schedules.  These  codes  provide  a record  of 
services  provided  and  the  amount  of  the  physician’s  time  involved  in 
various  activities  thereby  indicating  the  degree  of  interdisciplinary  activity. 
Physicians  at  Alexandra  have  made  further  revisions  to  these  codes  in 
order  to  make  them  more  usable.  They  could  be  standardized  and  used  by 
other  community  health  centres  to  track  types  of  service. 

Mapping  Health  Services/Building  Consensus 

The  Repositioning  project  has  developed  and  tested  tools  for  mapping 
existing  health  care  service  configurations  and  building  consensus  about 
how  best  to  reconfigure  services.  Because  they  were  sufficiently  tested 
these  tools  hold  promise  for  transferability.  The  mapping  tool  has 
particular  potential  for  use  by  health  care  organizations  to  map  existing 
services  and  identify  gaps. 
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Information  Technology  Evaluation  System 

A manual  suitable  for  use  by  other  primary  care  clinics/offices  for 
implementing  an  information  technology  evaluation  system  was  created 
and  revised  to  ensure  transferability  (Sunridge). 

Assessment  of  Immunization  Status  and  Attitudes  Toward 
Immunization 

An  instrument  used  to  assess  immunization  status  and  attitudes  towards 
immunization  was  used  in  Keeweetinok  Lakes  Regional  Health  Authority 
(KLRHA).  The  instrument  has  undergone  reasonable  validity  checks,  but 
to  date,  no  reliability  checks.  Reliability  and  validity  assessments  should 
be  conducted.  A detailed  report  of  the  project  team’s  experiences  and 
findings  using  this  instrument  (Van  Herk-Auger,  Denton,  Beyon,  and 
Brockway,  2000)  is  available  from  Alberta  and  Wellness  or  KLRHA. 

Assessment  of  Children’s  Mental  Health  Status 

A battery  of  instruments  has  been  compiled  to  assess  children's  mental 
health  status  in  the  C.O.P.E.  program.  Some  questions  used  in  this 
assessment  have  been  drawn  (with  permission)  from  the  National 
Longitudinal  Survey  for  Children  and  Youth  (NLSCY)  to  enable 
comparison  with  national  samples.  Data  collection  instruments  are 
transferable  to  other  school  sites  and  can  facilitate  comparison  with  other 
delivery  models. 

Assessment  of  Use  and  Attitudes  Towards  Accessibility  and  Use 

A tool  has  been  developed  to  measure  patient  perceptions  about 
accessibility  of  primary  care  service,  including  identification  of  barriers  to 
using  alternative  service.  The  tool  employs  a querying  strategy  (branching 
mechanism)  and  is  linked  directly  to  a database.  The  tools  are  applicable 
for  use  with  multicultural  populations  in  and  beyond  Emergency 
Department  (ED)  environments. 

The  process  that  accompanies  use  of  the  tool  highlighted  the  importance 
of  involving  key  stakeholders  (ED  staff,  physicians,  community  members) 
in  the  research  process  and  keeping  physicians  and  staff  apprised  of 
progress.  Cooperation  of  professionals  was  found  to  contribute 
significantly  to  the  achievement  of  project  goals.  The  project  also 
highlighted  an  unanticipated  finding — that  having  a "greeter"  in  the  ED 
eased  patients'  anxiety  about  being  in  the  Emergency  Department. 

Assessment  of  Need 

A series  of  data  collection  instruments  has  been  developed  to  assess 
needs  and  perspectives  of  providers  (oncologists,  family  physicians,  home 
care  coordinators),  and  patients  and  their  families.  Since  they  form  a 360- 
degree  assessment  they  have  potential  to  be  used  across  palliative 
environments  and  possibly  across  other  environments  as  well.  The  list  of 
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education  and  training  needs  identified  by  each  stakeholder  group  for 
themselves  and  for  other  care  providers  may  be  of  benefit  to  others. 

Communication  Template 

A standardized  template  has  been  developed  for  use  by  oncologists  and 
other  care  providers  (family  physicians,  home  care  coordinators)  to 
document/communicate  diagnostic  and  care  plan  information  for  palliative 
cancer  patients  (Calgary  Palliative  project).  This  template  has  applicability 
for  use  beyond  palliative  patients. 

Primary  Health  Care  Indicators 

The  list  of  50  indicators  and  the  instruments  based  on  these  will  be 
invaluable  to  others  interested  in  primary  care  evaluation.  These  tools, 
which  measure  integration,  access  and  quality,  consist  of  (a)  the  modified 
primary  care  assessment  survey,  (b)  the  access  instrument,  and  (c)  the 
comprehensiveness  instrument. 

Palliative  Brochure 

The  Palliative  Care  brochure  is  a print-based  resource  for  palliative 
services  in  the  CRHA.  Others  may  use  it  to  guide  the  development  of  their 
own  resource  directories. 

Palliative  Sedation  Clinical  Practice  Guideline 

A palliative  sedation  clinical  practice  guideline  (CPG)  is  available  along 
with  a method  to  test  its  adoption  and  appropriate  usage.  The  palliative 
sedation  guideline  and  a delerium  CPG  (currently  being  developed)  are 
available  for  use  in  palliative  care. 

Medication  Appropriateness 

A set  of  measures  has  been  assembled  to  assess  the  quality  of 
medication  use  including  medication  appropriateness,  compliance,  and 
health  status  (including  client  satisfaction).  As  well,  provider  characteristics 
and  behaviours,  and  processes  for  successful  team  building  have  been 
identified.  Documentation  to  support  improving  the  quality  of  medication 
use  in  the  community  (through  use  of  the  “Seamless  Solutions”  software 
package),  to  record  medication  review,  issues  identification  and  progress 
notes  for  follow-up  and  monitoring  is  also  available.  A template  for  day-to- 
day  Collective  activities  operationalizes  the  Joint  Statement  of  the 
Canadian  Medical  Association  and  the  Canadian  Pharmaceutical 
Association  on  enhancing  the  quality  of  drug  therapy  and  is  expanded  to 
include  collaboration  with  other  providers.  These  elements  may  be  used 
by  other  self-directed  professions  to  explore  their  own  collaborative 
practices  (e.g.,  roles  and  responsibilities,  patient/client  intake  and  flow 
criteria,  confidentiality  of  information,  record-keeping,  medications  and 
treatment  effectiveness,  and  case  conferencing  template). 
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Models 


Healthy  Families  Model 


The  Healthy  Families  program  was  implemented  in  two  different  ways — 
one  in  a large  urban  regional  health  authority,  the  other  in  a rural  health 
authority.  In  the  urban  pilot,  the  project  was  implemented  in  cooperation 
with  three  agencies  and  a multi-stakeholder  steering  committee.  Family 
Visitors  worked  closely  with  agency  staff  who  were  prime  recruiters  of  new 
mothers  and/or  pregnant  women.  Using  this  model  of  implementation,  the 
independent  evaluators  concluded  that  the  Capital  Health  Authority’s 
model  would  only  be  possible  where  there  existed  a mature  community 
agency  base  offering  a range  of  services  to  families  and  where  true 
partnerships  were  understood  and  practiced.  The  evaluators  suggested 
that  agency  partners  may  need  to  be  handpicked  since  significant 
commitment  in  terms  of  time  and  staffing  were  required  factors  considered 
critical  to  the  success  of  the  program. 

In  the  rural  pilot,  the  existing  regional  health  structure,  specifically  public 
health  nurses,  were  key  links  to  mothers  and  their  babies.  Family  Visitors 
then  accessed  the  mother  and  baby  through  the  public  health  nurse  after 
delivery  of  the  baby.  A multi-agency  advisory  committee  was  affiliated  with 
the  project  for  a short  time,  but  was  later  disbanded. 

Figure  8.  Capital  Health  Model 


Source:  Evaluation  of  the  Healthy  Families  Project,  2000 


Indications  are  that  the  model  of  implementation  for  Healthy  Families  in 
the  urban  environment,  that  is,  working  directly  through  existing  agencies 
connected  to  pregnant  women  and  mothers  with  babies,  was  more 
successful  in  reaching  women  and  their  children.  Both  pilots  emphasized 
that  the  training  modules  used  in  the  Healthy  Families  America  approach 
had  limitations  because  they  were  not  reflective  of  the  Canadian 
experience  (e.g.,  access  to  a variety  of  publicly-funded  support  services). 
A train  the  trainers  model  is  expected  to  facilitate  transfer  of  the  model  to 
local  needs,  and  reduce  the  expense  of  costly  training  in  the  U.S.  Ongoing 
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education  of  the  Family  Visitors  is  regarded  as  an  important  aspect  of  the 
program. 

Integrated  Service  Delivery  Model  for  Palliative  Care 

An  innovative  and  successful  model  of  community-based  palliative  care 
service  delivery  is  available  for  application  in  rural  and  possibly  other 
environments.  The  model  is  characterized  by  multidisciplinary  team 
delivery  of  services  coordinated  through  a Palliative  Care  Coordinator.  The 
model  of  holistic  24-hour  care  is  capacity  building  in  that  expertise  and 
information  are  shared  by  the  Consultation  Team  (Palliative  Care 
Physician  Consultant  and  Palliative  Care  Nurse  Consultant)  across 
provider  groups,  as  well  as  with  families/caregivers  and  community 
volunteers. 

The  manner  in  which  recommendations  and  advice  are  offered  by  the 
Consultation  Team  (as  “humble  experts”)  is  key  to  encouraging  and 
sustaining  changed  practice.  Each  encounter  with  the  Consultation  Team, 
whether  by  telephone  or  in  person,  is  an  opportunity  for  learning  about 
best  practice  related  to  palliative  care.  As  the  numbers  of  palliative 
patients  increase  and  best  practice  information  becomes  available,  these 
on-the-job  education  and  training  opportunities  become  increasingly 
important.  Bereavement  care  is  an  essential  component,  as  is  the  building 
of  a trusting  relationship  between  those  who  seek  consultation  and  those 
who  provide  it.  The  value  of  champions  at  senior  administrative  and 
management  levels,  as  well  as  at  nursing  and  community  levels  is  key. 

Strategies  for  case  management,  training  and  education,  and  best  practice 
are  available  from  the  project  (Lakeland  Palliative).  These  include 

(a)  Lakeland  Regional  Health  Authority  Policies  and  Procedures 
Manual  for  Palliative  Care  (outlines  scope  of  practice,  clinical 
practice  protocols), 

(b)  Volunteer  Manual  (outlines  content  and  application  of  skills  for 
volunteers), 

(c)  Orientation  Manual  for  Care  Providers, 

(d)  Service  Delivery  Model, 

(e)  Case  Management  System  (i.e.,  charting  procedures/forms),  and 

(f)  contents  of  training  sessions. 

Continuous  Quality  Improvement  Model 

A model  for  a Quality  Improvement  Process  was  developed  for  the 
selection  of  appropriate  initiatives  to  improve  the  quality  of  palliative  care 
in  the  Calgary  region.  That  model,  and  the  selection  criteria,  should  be  of 
interest  to  others. 
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Figure  9.  Logic  Model  for  Palliative  Care  Project  Goals  and  Objectives 


A Model  of  Shared  Mental  Health  Care 

A model  of  collaborative  practice  to  build  capacity  of  family  physicians  for 
appropriate  recognition,  treatment,  and  referral  of  patients  presenting 
mental  health  problems  in  a primary  care  setting  was  successfully 
implemented  in  the  Calgary  Regional  Health  Authority.  Before  the  model 
can  be  exported  to  the  national  level,  however,  the  independent  evaluator 
suggests  that  greater  numbers  of  physicians  within  the  region  need  to  pilot 
the  model.  Twenty-four  self-selected  family  physicians  who  exhibited  an 
interest  in  improving  their  understanding  and  delivery  of  mental  health 
services  were  involved  in  the  pilot  (there  are  800  primary  practice 
physicians  in  Calgary).  Results  to  date  are  promising  that  collaborative 
practice  between  psychiatrists  and  family  physicians  is  beneficial  to  both 
health  providers  and  clients/patients. 

A Model  for  School-based  Children’s  Mental  Health  Outreach 
Program 

A service  delivery  model  (developed  by  the  C.O.P.E.  program)  is 
transferable  to  school  systems  almost  everywhere.  C.O.P.E.  is  a 
successful  model  of  community-based  service  delivery  to  identify  and 
address  children’s  mental  health  disorders — especially  important  for 
families  living  in  high-risk  communities.  The  intake  referral  and  diagnosis 
systems  are  exemplary  models  of  collaboration  between  health  and 
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education  systems.  The  model  piloted  in  the  Calgary  region  involved 
medical  intervention  teams  working  in  collaboration  with  school-based 
professionals.  The  pilot  demonstrated  how  parents  become  better  able  to 
identify  and  articulate  their  children’s  mental,  emotional,  and  behavioural 
problems.  The  C.O.P.E.  model  also  demonstrated  how  communities  can 
assume  greater  responsibility  to  address  children’s  emotional  and 
behavioural  disorders.  Complex  cases  may  not  be  appropriate  for  the 
C.O.P.E.  intervention.  However,  C.O.P.E.  serves  a useful  triage  function 
to  identify  and  direct  to  appropriate  levels  of  care.  Supportive 
reimbursement  structures  are  required.  A fee-for-service  funding  structure 
does  not  appear  to  support  collaborative  initiatives  like  C.O.P.E.  (school- 
based  mental  health  programs).  Neither  does  differential  payment  for 
psychiatrists  and  pediatricians.  Information  sharing  across  health  and 
education  sectors  needs  to  be  facilitated  for  ease  of  implementation  of  the 
model. 

Program  for  Lifestyle  Risk  Reduction  (Cholesterol) 

Program  materials  for  cholesterol  reduction  include  program  planning 
information,  patient  manual,  instructor  manual,  PowerPoint  presentation 
slides,  and  complete  references  and  literature  search  information. 
Materials  will  be  published  on  CD-ROM  for  transfer  to  other  sites  across 
Canada. 

Model  for  Building  a Healthy  Community  Coalition 

The  Healthy  Okotoks  Coalition  evaluation  results  can  be  transferred  to  and 
used  as  a guide  to  creating  other  healthy  community  coalitions.  These 
results  provide  an  understanding  of  what  it  takes  to  develop  a healthy 
communities  coalition.  The  evaluation  log  can  be  used  as  a template  to 
track  process  and  progress. 
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Figure  10.  Conceptual  Model  of  the  Process  of  Creating  a Healthy 
Community  in  Okotoks,  Alberta. 


Source:  Hodson  et.  al.,  2000 


Model  to  Establish  Primary  Health  Care  Collectives  (Physicians, 
Pharmacists,  Home  Care  Case  Managers) 

The  Collectives  project  created  a template  or  model  for  day-to-day 
Collectives  activity  which  has  operationalized  the  Joint  Statement  of  the 
Canadian  Medical  Association  and  Canadian  Pharmaceutical  Association 
to  enhance  quality  of  drug  therapy  and  expanded  the  model  to  include 
collaboration  with  other  providers.  Other  self-directed  professions  may  find 
this  template  useful  to  explore  dimensions  of  their  own  collaborative 
practice.  Several  learnings  are  transferable  to  others  interested  in  forming 
collectives  of  providers. 

1 . Involve  all  professional  groups  in  the  team  of  investigators  who 
originally  design  the  project. 

2.  Involve  additional  key  stakeholders  in  the  Steering  Committee. 

3.  Include  a control  group  in  the  project  research  design. 

4.  Provide  ongoing  facilitator  consultation  for  the  intervention  groups  and 
individuals. 

5.  Provide  a high  level  of  communication. 

6.  Plan  longer  timelines  for  patient/client  recruitment  and  follow  up. 
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7.  Screen  all  potential  Collectives  participants. 

8.  Do  collaborative  team-building  before  taking  on  clinical  responsibility. 

9.  Focus  on  high  users  of  services  with  chronic,  multiple  medical 
conditions. 

10.  Pay  professionals  at  market  value  for  the  collaborative  activity  needed 
by  patients/clients  with  complex  health  needs  to  support  better 
outcomes. 

1 1 . Use  full  time  home  care  and  pharmacist  resources  whose  time  is 
dedicated  to  the  collaborative  practice. 

12.  Begin  the  collaborative  practice  together. 

Critical  success  factors  for  implementing  research  to  investigate 
functioning  of  the  Collectives  include  the  following. 

1 . Involve  all  professionals  in  the  team  of  investigators  who  originally 
design  the  project  so  that  concepts,  roles,  responsibilities,  and 
perspectives  are  clear. 

2.  involve  additional  stakeholders  in  the  Steering  Committee  (research 
and  policy  bodies). 

3.  Provide  ongoing  facilitations  for  the  intervention  groups  and 
individuals. 

4.  Maintain  communication  among  team  members  and  between  the 
project  and  the  teams  for  motivation,  early  detection  of  provider 
dissatisfaction,  problems,  and  to  promote  accountability. 

5.  Include  a control  group  in  project  design. 

6.  Allow  for  longer  timelines  for  recruitment  (one  year)  and  follow  up  (two 
years)  when  there  is  only  one  physician  per  Collective. 

Economic  Analysis 

The  development  of  a mechanism  to  assess  cost-effectiveness  within  a 
primary  health  care  setting  is  promising  given  the  struggle  many  projects 
have  experienced  in  this  regard.  The  CPPOA  model  (Cost  Procedure 
Process  Outcome  Assessment)  piloted  successfully  in  three  projects  has 
previously  been  tested  in  various  human  services  programs.  (For  more 
information  on  this  model,  see  the  Cost-Effective  dimension.) 

Sustainability  Analysis 

A pilot  tool  was  developed  to  assess  sustainability  of  human  service 
programs  to  sustain  long-term  outcomes.  The  tool  will  require  more  validity 
and  reliability  testing  before  it  has  broad  applicability  in  human  service 
programming.  However,  to-date,  it  has  been  useful  in  assisting  initiatives 
with  strategic  planning  in  several  community  development  projects. 
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Figure  11.  SDR  (Self-Determination  Self-Reliance)  Growth  Model 
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Model  to  Reconfigure  Health  Services  Delivery 

A Total  Health  Process  strategy  was  developed  that  might  work  as  a tool 
to  create  a visual  map  of  the  linkages  between  services,  providers,  and 
populations  in  a primary  health  care  system.  As  an  analysis  mechanism  it 
has  potential  application  for  use  in  the  reconfiguration  of  health  service 
delivery  systems.  However,  before  it  can  be  confirmed  that  the  emerging 
model  is  viable  and  of  benefit  to  providers  and  users,  it  will  be  necessary 
to  pilot  it.  Three  possible  anchors  or  orientations  to  organizing  the  delivery 
of  services  are  suggested  by  the  model: 

(a)  a primary  health  care  anchor  (according  to  geographic,  cultural, 
special  interest,  socially/clinically  complex  patients); 

(b)  a systems  anchor  (support  after  hour  care,  specialty  episodic  care 
and  administrative/infrastructure  support);  and 

(c)  a preferred  provider  arrangements  anchor  (single  specialty,  multi- 
specialty including  facilities,  equipment  and  supplies). 

To  determine  impact  on  populations  and  cost  savings  to  the  health  system 
over  the  long  term,  comprehensive  tracking  and  analysis  of  the  cost  and 
effectiveness  of  the  model  would  need  to  be  conducted.  Piloting  will 
require  leadership  and  resource  support  at  senior  levels  and  among 
providers  at  all  levels.  While  a new  model  for  health  system  delivery  has 
been  developed,  it  has  not  been  tested  and  therefore  its  value  is 
uncertain. 
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Community  Health  Centre  Models 

Implementation  of  another  community  health  centre  (CHC)  in  Alberta 
(Airdrie  Regional  Health  Centre)  has  resulted  in  a set  of  recommendations 
for  other  jurisdictions  contemplating  the  development  of  a health  centre 
within  a regional  context.  In  contrast  to  the  Northeast  CHC  which  is 
governed  by  the  Capital  Health  Authority,  the  governance  model  of  the 
Airdrie  centre  involved  three  key  stakeholder  groups  (Calgary  Regional 
Health  Authority,  Rocky  View  Municipal  District,  and  the  City  of  Airdrie). 
Efforts  to  operationalize  the  Airdrie  Regional  Health  Centre  during  a time 
of  system  reorganization  involved  the  unique  feature  of  a partnership 
arrangement  of  co-location  with  three  community  agencies. 
Recommendations  for  implementation  of  a community  health  centre  within 
this  context  include  the  following. 

1 . Ensure  a common  and  clear  vision. 

2.  Take  a position  on  the  primary  care/primary  health  care  debate. 

3.  Be  tangible  and  keep  things  simple. 

4.  Build  on  existing  service  strengths  and  models  rather  than  building  a 
new  one. 

5.  Assign  a task  force  with  a mandate  to  start  with  a broad  focus,  then 
move  quickly  to  specific  tasks  and  addressing  specific,  tangible  issues. 

6.  Engage  a whole  range  of  people. 

7.  Work  from  a common  base,  and  find  and  build  upon  common  interests. 

8.  Focus  on  building  capacities. 

9.  Focus  on  internal  integration  before  extending  out  to  compass  players 
outside  the  organization. 

10.  Be  willing  to  take  risks. 

1 1 . Look  to  build  a system  of  services,  not  stand  alone  services. 

12.  For  effective  consultation  with  or  involvement  of  the  community,  strong 
leadership  and  a will  to  consult  are  needed. 

13.  For  community  partnerships  to  work,  open,  meaningful,  and 
transparent  processes  are  needed. 

14.  When  embarking  on  community  consultation,  be  sure  you  know  which 
questions  to  ask  and  what  you  want  back  from  the  community. 

15.  Determine  which  core  or  standard  services  will  be  provided. 

16.  Be  aware  of  how  political  situations  really  are  and  be  smart  about 
dealing  with  political  issues. 

Findings  from  the  evaluation  of  an  inner  city  CHC  (8th  and  8th  Health 
Centre)  that  provides  immediate  medical  care  24  hours  per  day,  seven 
days  a week,  mental  health  services,  public  health  services,  home  care 
services  and  community  development  and  liaison  suggests  that  for  others 
attempting  to  establish  a CHC  within  large  regional  health  authorities, 
more  than  one  centre  should  be  planned.  Multi-purpose  centres  should  be 
designed  as  alternatives  to  emergency  departments  since  the  majority  of 
clients  can  be  treated  at  primary  health  care  centres.  The  independent 
evaluator  suggests  that  such  centres  should  be  geographically  based  and 
targeted  to  the  needs  of  the  communities  they  serve.  An  ideal  model  might 
be  independent  private  practices  linked  to  a multi-centre  hub.  The 
population  base  should  be  identified  and  a maximum  size  set.  Holistic, 
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preventative  and  comprehensive  care  should  supercede  urgent  care.  Also, 
a community-based  governance  model  should  be  established — not  a 
community  board  but  a broader  public  organization  to  create  better 
accountability. 

Learning  was  derived  from  evaluation  of  yet  another  CHC  (the  Alexandra 
Health  Centre  near  Calgary’s  inner  city).  Here  information  from  assessing 
the  functioning  of  the  administration  and  board  could  be  generalized  to 
other  centres.  A shift  to  recruit  board  members  from  outside  the 
community  originally  served  by  the  Alexandra  Health  Centre  has  resulted 
in  some  clients  feeling  that  the  Centre  is  dislocated  from  its  community. 
Furthermore,  staff  turnover  suggests  that  dissatisfaction  stems  from 
insufficient  involvement  of  staff  in  important  decision-making  processes, 
role  confusion,  and  the  tendency  for  some  staff  to  assume  multiple  roles. 
These  findings  shed  additional  insight  into  governance  and  administrative 
issues  that  may  arise  in  response  to  a shifting  population  base,  the 
changing  demographic  of  the  population  intended  to  be  served  by  the 
Centre,  or  both.  Changes  in  the  community  context  may  necessitate 
revisiting  an  assessment  of  community  needs  and  the  selection  of 
services  to  meet  those  needs. 

Common  across  evaluations  of  the  four  CHCs  was  the  need  for  a 
comprehensive  information  management  system  to  track  clients/patients. 
In  addition,  information  systems  needed  to  be  able  to  interface  with 
regional  and  provincial  databases.  Most  projects  (even  those  beyond 
evaluations  of  community  health  centres)  discovered  the  incompatibility 
among  existing  database  systems,  whether  regional  or  local.  Second,  a 
common  need  across  CHCs  was  the  development  of  mechanisms  and 
systems  to  capture  relevant  and  sufficient  information  to  enable  cost 
analyses  with  the  potential  to  contribute  to  an  assessment  of  effect  on 
populations  (health  status).  The  Northeast  Centre  has  identified  relevant 
questions  to  guide  this  process  and  is  in  the  process  of  developing  an 
interface  with  regional  databases.  An  econometric  methodology  was 
developed  and  tested  using  three  time  periods  as  comparators  at  the  8th 
and  8th  Health  Centre.  This  approach  may  be  useful  to  others  wishing  to 
assess  the  cost/feasibility  of  specific  services  across  varying  models  of 
primary  health  care.24 


Implementation  Strategies/Processes 

Introduction  of  a Nurse  Practitioner 


Two  approaches  were  used  in  the  PHC  project  to  introduce  a Nurse 
Practitioner  into  existing  health  facilities.  In  the  first  case  a community 
development  approach  was  used  within  a Healthy  Communities 
framework.  This  approach  resulted  in  a renewed  partnership  between  the 
region  and  community  residents,  who  were  motivated  by  the  closure  of 
their  local  hospital.  The  service  delivery  model  used  in  this  rural  site 


24  See  Auld,  C.  (2000),  Cost-effectiveness  analysis  of  the  8th  and  8th  Clinic.  University  of  Calgary. 
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(Elnora)  may  be  transferable  to  other  rural  communities  but  circumstances 
would  vary  from  one  site  to  the  next.  In  the  Elnora  Centre  providers  were 
already  working  together.  The  interpersonal  skills  of  the  Nurse  Practitioner 
were  also  key  to  facilitating  greater  team  integration  and  community 
acceptance  of  the  services.  The  independent  evaluators  viewed  these  two 
factors  as  significant  and  reminded  that,  at  present,  regulatory  and  funding 
mechanisms  within  Alberta  may  limit  transferability  of  the  Nurse 
Practitioner  concept.  Currently,  funding  of  NPs  through  regional  budgets 
while  continuing  to  fund  physicians  through  provincial  budgets  limits  the 
feasibility  of  NPs  as  a less  costly  alternative  to  physicians.  Current 
professional  regulations  regarding  scope  of  practice  would  need  to  be 
reviewed  to  allow  NPs  to  practice  more  readily  in  a variety  of  settings  with 
a variety  of  providers. 

In  the  second  case,  a focus  on  team  building  and  collaborative 
development  of  ten  treatment  conditions  for  the  NP  (clinical  practice 
guideline  process)  facilitated  successful  implementation  of  the  NP  role  into 
the  inner  city  Health  Centre  (CUPS).  Support  of  the  region  (CRHA), 

Centre  board,  administration,  and  physicians  was  considered  critical. 
Evidence  supports  the  transfer  and  utility  of  NP  extended  practice  skills 
from  isolated  northern  settings  to  urban  settings.  This  model  of 
implementation  (and  the  10  treatment  conditions)  is  expected  to  be 
transferable  to  other  settings. 

Introduction  of  a Settlement  Nurse  on  Metis  Settlements 

A process  for  introducing  a Settlement  Nurse  to  a Metis  Settlement  was 
successfully  implemented.  The  project  demonstrated  the  importance  of 
willingness  on  the  part  of  the  regional  executive  to  engage  directly  with 
Settlement  leaders  in  a process  of  community  building,  not  just  “delivering 
services.”  The  model  includes  the  establishment  of  Aboriginal  Liaison 
Worker  and  Community  Health  representatives  for  each  Settlement, 
identification  of  health  services  based  on  a community  needs  survey, 
consultation  with  the  community  on  the  introduction  of  a Settlement  Nurse 
(including  the  hiring  and  selection  of  the  Nurse),  and  strong  commitment  of 
senior  management  in  the  region.  The  first,  and  most  critical,  strategy  to 
increase  knowledge  and  awareness  of  the  residents  is  to  gain  their 
acceptance  and  trust. 

Stakeholder  Involvement  in  Planning  Primary  Health  Services 

The  East  Central  project  focused  its  project  on  the  transformation  of  its 
existing  system  of  health  service  delivery  to  a primary  health  services 
system  built  on  a collaborative  planning  process  involving  four  rural 
communities  across  the  region.  Significant  learning  has  been  derived  from 
this  planning  process  that  will  be  useful  to  others. 


A Primary  Health  System  (PHS)  enables  people  to  take  charge  of  their  own  health 
and  to  take  an  active  part  in  the  planning,  policy  making  and  delivery  of  health 
services  in  their  community  (East  Central  Health). 
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The  commitment  to  integrated  planning  and  development  of  capacity  for 
primary  health  services  within  the  Steering  Committee  resulted  in  a type  of 
leadership  and  forum  for  discussion  that  could  be  modeled  in  a variety  of 
other  system-wide  initiatives  in  health  and  social  services.  Transferable 
lessons  include  the  following. 

1 . Recognize  that  communities  have  a kind  of  expertise  that  health 
regions  need  at  the  table  when  planning  for  integrated  primary  health 
services. 

2.  Encourage  health  system  professionals  to  focus  beyond  hospitals  and 
disease.  The  community  may  be  more  interested  in  health  promotion 
and  prevention  than  you  assume. 

3.  Insist  on  an  open,  respectful,  and  partnership-oriented  approach.  It 
fosters  good  credibility  and  good  relations  with  each  community. 

4.  Be  careful  to  not  over  formalize.  Allow  the  process  to  follow  a natural 
process  in  the  community  and  avoid  technical/organizational  jargon. 

Introducing  Telehealth  to  Northern  Communities 

Learning  from  the  telehealth  project  indicates  that  readiness  to  implement 
telehealth  requires  adequate  staffing;  clearly  defined  roles  and 
responsibilities  for  staff;  adequate  training  for  staff,  providers  and  clients; 
and  properly  functioning  equipment. 

Implementing  Special  Research  Approaches  (Participatory  Action 
Research) 

The  Brooks  Cares  project  highlighted  that  a participatory  action  research 
(PAR)  strategy  has  merit  as  a successful  strategy  to  empower 
disenfranchised  or  disengaged  people  (i.e.,  in  building  capacity  of 
community  members).  Several  learnings  that  are  relevant  to  others 
attempting  to  implement  PAR  projects  emerged. 

1 . Focus  on  issues  that  are  real  issues  for  community  members,  not  lofty, 
unclear  goals. 

2.  Clarify  project  goals  and  outcomes  to  community  members  before 
starting  the  PAR  process. 

3.  Develop  realistic  project  outcomes.  Examples  might  include 

(a)  improved  mental,  physical  and  social  health  status, 

(b)  improved  social  support, 

(c)  increased  integration  of  services,  and 

(d)  reduced  drug,  alcohol,  and  crime  rates. 

4.  Develop  appropriate  indicators  of  success.  Examples  might  include 

(a)  increased  knowledge  and  abilities  to  use  support  systems 
appropriately, 

(b)  increased  trust  and  communication  among  and  between  citizens 
and  human  services  professionals, 

(c)  increased  opportunities  for  learning,  employment,  recreation  and 
spiritual  development, 
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(d)  widespread  understanding  of  goals  and  processes,  and 

(e)  substantial  voluntary  participation. 

5.  Ensure  that  the  implementation  team  is  clear  from  the  outset  what  the 
project  is  about.  The  team  must  be  able  to  articulate  what  the  purpose 
and  process  is.  Team  members  must  be  a good  match  with  the 
community. 

6.  Consider  necessary  skills  and  talents  (e.g.9  leadership)  of  an 
experienced  community  development  worker  and  a steering  group  with 
broad  representation  of  organizations’  and  citizens'  needs  which 
shares  the  same  vision  of  development  and  participatory  processes. 

7.  Prepare  the  community  (politically,  socially)  for  the  PAR  process. 
Introduce  the  PAR  to  the  community  in  a way  that  makes  it  real  and 
readily  applicable  to  daily  life. 

8.  Grassroots  support  is  imperative.  Work  with  people  at  their  own  level. 

9.  Support  of  existing  agencies  is  required.  Draw  them  in  as  advocates 
and/or  supporters. 

10.  Expect  challenges/barriers  (e.g.,  skepticism,  jealousy,  vested  interests, 
fear,  communication  limitations — individual  and  systemic). 

1 1 . Funding  support  for  a successful  PAR  process  may  be  required  for  3 
to  5 years  to  establish  a strong  core  group  of  people  who  are  engaged 
and  committed  to  the  process  over  the  long  term. 

Summary  of  Learning 

1 . In  communication  between  projects  (even  similar  projects)  it  became 
obvious  that  context  of  a project  was  extremely  important.  For 
example,  community  health  centres  met  to  discuss  the  potential  for 
common  client  satisfaction  instruments.  While  these  talks  were 
informative  in  a general  way,  each  Centre  was  sufficiently  and  uniquely 
different  in  the  type  of  audience  it  served  and  the  approaches  it  used, 
that  a decision  was  made  not  be  select  a common  instrument.  It  is 
important  that  information  from  all  projects  be  considered  in  light  of  the 
setting  and  circumstances  of  each  project.  Each  project  produced  a 
project  report  and  an  independent  evaluation  report.  A bibliography  of 
evaluation  reports  is  attached  as  an  Appendix  to  this  report.  A number 
of  evaluation  approaches  were  used  and  they  will  have  applicability  in 
the  assessment  of  primary  health  care  initiatives. 

2.  Many  of  the  tools  developed  are  not  standardized  instruments. 
However,  a number  of  them  will  benefit  from  further  refinement.  Others 
may  wish  to  use  these  instruments  thereby  assisting  in  establishing 
indicators  of  validity  and  reliability.  Each  iteration  will  greatly  assist  in 
adding  value  to  the  transferability  of  instruments  and  further  the  return 
on  investment  already  made  in  their  development. 


Tools  include 

(a)  Instruments  to  provide  client  information; 

(b)  Codes  for  shadow  billing; 
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(c)  Tools  for  mapping  health  care  service  configurations  and  building 
consensus  about  how  to  reconfigure  services; 

(d)  An  information  technology  evaluation  system; 

(e)  An  instrument  to  assess  immunization  status  and  attitudes; 

(f)  An  instrument  to  assess  children’s  mental  health  status; 

(g)  A tool  to  measure  patient  perceptions  about  accessibility  of  primary 
care  service; 

(h)  Instruments  to  assess  needs  and  perspectives  of  providers  and 
patients  and  their  families; 

(i)  A template  to  document/communicate  diagnostic  and  care  plan 
information  for  palliative  cancer  patients; 

(j)  Tools  which  measure  integration,  access,  and  quality; 

(k)  A palliative  care  brochure; 

(l)  A palliative  sedation  clinical  practice  guideline;  and 

(m)  Measures  to  assess  the  quality  of  medication  use. 

Models  include 

(a)  Healthy  Families  Model, 

(b)  Integrated  Service  Delivery  Model  for  Palliative  Care, 

(c)  Continuous  Quality  Improvement  Model, 

(d)  A Model  of  Shared  Mental  Health  Care, 

(e)  A Model  for  School-based  Children’s  Mental  Health  Outreach 
Program, 

(f)  Program  for  Lifestyle  Risk  Reduction  (Cholesterol), 

(g)  Model  for  Building  a Healthy  Community  Coalition, 

(h)  Model  to  Establish  Primary  Health  Care  Collective  (Physicians, 
Pharmacists,  Home  Care  Case  Managers), 

(i)  Economic  Analysis, 

(j)  Sustainability  Analysis, 

(k)  Model  to  Reconfigure  Health  Services  Delivery,  and 

(l)  Community  Health  Centre  Models. 

3.  Economic  evaluation  models  used  in  the  Alberta  projects  were  derived 
from  tested  approaches  grounded  in  established  theory  of  cost 
analysis,  cost-effectiveness,  cost-minimization,  and  cost-benefit. 
However,  individual  projects  invariably  had  some  difficulty  in  applying 
these  models.  In-servicing  in  economic  evaluation  may  alleviate  some 
of  the  "fear"  associated  with  economic  evaluations. 

4.  Some  common  process  themes  across  Alberta  projects  emerged: 

a)  Stakeholder  participation  in  setting  objectives  for  initiatives  goes  a 
long  way  to  establish  cooperation  and  willingness  to  participate. 

b)  Setting  realistic  objectives  is  critical. 

c)  Clearly  establish  roles  of  participants. 

d)  Insist  on  open  communication  protocols. 

e)  Assess  the  skills  and  talents  required  to  advance  the  initiative. 

f)  Establish  comparison  or  control  groups  wherever  possible. 

g)  Establish  appropriate  indicators  of  success — this  may  take  some 
consensus  building. 

h)  Choose  standardized  measurement  instruments  where  possible. 

i)  Consider  remuneration  to  professionals  for  participation  in  projects 
that  take  them  away  from  daily  practice  patterns. 
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j)  Allow  for  ethics  review  and  participant  recruitment  delays. 

k)  Build  on  existing  service  models  where  possible. 

l)  Innovation  and  new  ideas  rarely  happen  without  some  risk. 

m)  Recognize  the  importance  of  communities  (public  and  collegial). 
Community  consultation  is  important — but  have  a clearly 
articulated  consultation  agenda. 

n)  Assess  the  internal  and  external  environment  from  the  perspective 
of  support. 

o)  Work  from  a position  of  being  informed,  assess  information 
requirements.  Determine  barriers  to  accessing  of  information. 

p)  Work  from  a common  definition  of  primary  health  care. 

q)  In  multiple  project  implementations  meet  with  other  projects  to 
share  experiences. 

r)  Expect  some  deviation  in  plans.  Be  reasonably  flexible  in 
establishing  expectations. 

s)  While  accountability  is  essential,  being  too  formal  in  approach  can 
create  artificial  and  unnecessary  barriers  to  participation  and 
action. 
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Implications 


Alberta’s  project  was  a major  effort  to  learn  more  about  advancing  primary 
health  care  in  Alberta.  Comprehensive  evaluation  of  the  27  projects  has 
resulted  in  significant  contributions  to  addressing  questions  identified  by 
the  Health  Transition  Fund — access,  quality,  integration,  health  status, 
cost-effectiveness,  and  transferability. 

A consultation  process  held  in  the  initial  stages  of  Alberta’s  project 
resulted  in  the  identification  of  essential  elements  of  primary  health  care. 
These  include  community  involvement,  accessibility,  availability, 
comprehensiveness,  integration/coordination,  determinants  of  health 
approach,  appropriate  use  of  technology,  accountability,  affordability,  and 
ongoing  evaluation  to  improve  quality,  delivery,  and  outcomes.  Analysis  of 
findings,  perhaps  not  surprisingly,  indicates  implications  for  advancing 
primary  health  care  in  Alberta  that  address  most  of  these  elements. 

Project  learning  points  toward  policy  change  in  six  areas:  integrated 
service  delivery,  information  sharing,  methods  of  payment/incentives, 
urgent/emergency  care,  prevention/promotion  and  rural  health.  Learning 
also  points  toward  support  for  policy  change  in  three  areas:  change 
management,  regional/community  relations,  and  monitoring  and 
evaluation.  Each  is  addressed  below. 

Integrated  Service  Delivery 

A major  theme  across  projects  was  integration.  A great  many  projects 
exemplified  integration  through  collaborative  planning  processes  that 
involved  consultation  with  stakeholders,  health  professionals,  inter-agency 
and  cross-sectoral  partnerships,  and  some  negotiation  between 
professional  organizations.  Successful  integration  depended  on  the 
willingness  of  health  professional  to  cooperate — as  one  evaluator 
suggested,  moving  outside  their  respective  “zones  of  comfort.”  Trust  was 
considered  central  to  relationship  development  and  team  building. 
Leadership  (often  by  physicians  and  other  health  professionals),  was  key 
in  catalyzing  the  participation  of  others. 

Working  Together 

Development  of  practice  guidelines  and  investigating  multiple  perspectives 
about  what  constitutes  best  practice,  was  a time-intensive  and  often 
difficult  process  that  required  commitment  in  terms  of  time  and  energy. 
However,  time  devoted  to  sharing  perspectives  appears  to  result  in 
creative  insights  and  greater  appreciation  of  the  value  others  contribute  to 
the  continuum  of  care,  and  how  health  professionals  can  work  to 
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complement  each  other’s  skills.  Time  and  again  we  heard  that  health 
professionals  had  never  really  had  an  opportunity  to  talk  and  ask 
questions  about  each  other’s  practice.  We  also  heard  that  it  may  be  that 
physicians  find  it  the  most  difficult  to  learn  to  work  differently.  Their 
working  culture  is  one  of  independent  decision-making,  rather  than 
collaborative  development  of  practice-based  guidelines,  particularly  with 
the  participation  of  multiple  care  providers  who  generally  work  at  different 
levels  in  the  care  cycle. 

Client’s  Role 

The  patient/client  plays  an  important  role  in  an  integrated  health  system. 
Projects  demonstrated  several  examples  where  stakeholder  involvement 
was  an  integral  part  of  the  planning  process.  Primary  health  care  includes 
the  mobilization  of  community  resources,  and  increased  community 
responsibility  for  the  provision  of  holistic,  quality  care  that  addresses  the 
physical,  mental,  and  spiritual  needs  of  the  patient.  Stakeholder 
involvement  supports  integration  of  health  services. 

Continuity 

Several  projects  demonstrated  that  interagency  and  cross-sectoral 
collaboration  are  important  for  continuity  of  care  (C.O.P.E.,  Healthy 
Families,  Palliative  Care).  Some  projects  highlighted  that  the  next  level  of 
care  is  not  available,  even  when  access  to  diagnostic  services,  for 
example,  is  improved.  Attempts  to  provide  or  create  an  integrated  service 
delivery  system  emphasized  the  gaps  that  currently  exist,  and  challenges 
that  prevent  further  efforts  to  achieve  integration  (e.g.,  lack  of  provincial 
government  approval  for  the  continuing  role  of  the  Nurse  Practitioner). 
However,  some  progress  has  been  made.  For  example,  the  agreement  in 
principle  between  the  Alberta  College  of  Physicians  and  Surgeons  and 
Department  of  Family  Medicine  in  the  Faculty  of  Medicine  and  Dentistry 
about  enrollment  of  patients  in  collaborative  practice  by  members  other 
than  physicians.  This  move  has  potential  to  increase  the  cost- 
effectiveness  of  collaborative  practice  model  used  to  improve  medication 
appropriateness  and  medication  compliance.  Collaborative  practice 
models,  in  general,  have  implications  for  university  preparation  and 
training  of  health  professionals. 

Information  Sharing 

Critical  to  integration  is  information  sharing  among  stakeholders — 
providers,  clients,  public,  agencies,  regional  administration,  and  provincial 
administration.  There  appears  to  be  a need  for  clinically-integrated 
databases  where  on-line  real  time  information  is  simultaneously  available 
to  care  providers.  Such  information  sharing  would  support  inter- 
disciplinary collaboration  (e.g.,  through  electronic  real-time  health 
records).  Structures  and  processes  are  required  to  support  the  creation 
and  ease  of  use  of  such  databases. 
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Standardized  Information 


Standardization  of  information  sharing  such  as  common  communication 
mechanisms  (e.g.,  templates)  and  reporting  formats  (easily  accessible 
electronically)  can  enhance  information  sharing.  Communication 
Passports  (for  patients)  have  proven  successful  in  transfering  information 
from  oncology  specialists  to  community-based  family  physicians. 

Linkages 

Many  projects  illustrated  that  the  physician  is  often  an  important,  if  not 
presently  the  most  important,  link  in  the  information-sharing  process. 
Projects  also  demonstrated  that  a host  of  other  care  providers,  besides 
physicians,  have  an  integral  role  to  play  in  creating  linkages  to  information 
and  to  services.  Community  human  service  agencies,  for  example,  can 
provide  an  excellent  source  of  advice  and  information  on  health-related 
issues. 

Methods  of  Payment/Incentives 

A clear  message  from  several  projects  confirms  that  current  methods  of 
payment  and  incentives  for  health  providers  discourage  involvement  in 
collaborative  activity.  For  example,  if  the  intent  is  to  have  providers  work 
more  closely  together,  the  issue  of  differential  reimbursement  needs  to  be 
addressed  (e.g.,  psychiatrists  can  bill  for  consults  without  the  patient  being 
present  while  family  physicians  cannot).  It  may  be  necessary,  as  the 
C.O.P.E.,  Multidisciplinary  Teams,  and  Collectives  projects  illustrated,  to 
reimburse  physicians  and  others  (pharmacists,  home  care  case 
managers)  for  time  to  engage  in  team  planning  exercises  for  patients  with 
complex  health  needs.  As  benefits  are  realized  for  both  patient  and 
provider,  increasing  numbers  of  physicians/providers  will  support  alternate 
forms  of  organizing  and  delivering  care. 

Payment  Schedules 

Specifically,  conventional  fee-for-service  models  do  not  recognize  the  non- 
billable time  a physician  may  spend  with  clients  in  a primary  health  care 
setting  (e.g.,  telephone  consultations,  and  holistic  care).  In  the  case  of 
integrating  physician  and  pharmacist  services,  a re-examination  of  the 
payment  structure  to  include  “cognitive  services”  for  pharmacists  involved 
in  team  care  for  complex,  high  needs  patients  is  necessary  (i.e.,  to 
conduct  medication  reviews,  identify  drug-related  and  medication 
compliance  issues,  and  assist  in  monitoring  and  follow  up).  The  following 
table  indicates  specific  areas  for  extended  scopes  of  practice. 
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Table  12.  Publicly  and  Non-publicly-funded  Services 


Publicly-funded  Service  Non-Publicly-funded  Services 

Training  for  team  care 


Physicians 

Diagnosis/treatment  of  disease 

Home  care  case  manager 

Home  care  assessments  and  coordination 
of  services 

Pharmacists 

Medication  dispensing  services 


Participation  in  case  conferences  for 
patients/clients  at  risk  for  medication 
problems 

Medication  monitoring 

Follow  up  and  referral  for  patients  not  on 
home  care 

Medication  reviews 

Documentation  of  medication  issues 


Two  different  types  of  reimbursement  (contract  and  fee-for-service)  were 
used  with  physicians  in  the  community  health  centres  studied  within  the 
PHC  project.  Further  investigation  is  required  into  the  influence  of  these 
different  schemes  on  scopes  of  practice,  incentive  to  participate  in 
collaborative  activities  (those  that  would  indicate  movement  towards  an 
integrated  health  system),  appropriate  shadow  billing  codes,  etc. 

Reimbursement  for  consultation  time  by  physicians  may  also  need  to  be 
considered  more  broadly  than  consultation  with  other  health  providers.  For 
example,  the  East  Central  project  highlighted  the  need  for  increased 
involvement  of  physicians  in  community-based  decision-making  processes 
(e.g.,  in  organizing  primary  health  services  in  East  Central  region).  The 
degree  to  which  physicians  are  reimbursed  for  currently  non-billable 
activities  will  influence  their  willingness  to  be  engaged  in  such  processes. 

Urgent  Care  Versus  Emergency  Care 

The  issue  of  urgent  versus  emergency  care  highlights  the  importance  of 
clarifying  the  impact  on  resources  of  some  “first  contact”  services  as 
opposed  to  others.  For  example,  the  Emergency  Department  survey 
project  suggests  that  users  choose  the  ED  when  they  consider  the  need 
urgent,  generally  following  attempts  to  contact  another  medical 
professional  such  as  a family  physician,  a walk-in  clinic,  or  consultation 
with  a non-professional.  Worry  and  a sense  of  urgency  serve  as  key 
drivers.  These  findings  suggest  that  people  choose  to  use  the  ED  for  good 
reason,  rather  than  out  of  convenience  or  proximity  to  home.  These 
findings  indicated  that  understanding  clients’  “mindsets”  for  using 
particular  services  helps  distinguish  between  perspectives  of  emergency 
and  urgency — implying  a greater  need  for  triage  (primarily  telephone)  with 
assistance  in  multiple  languages. 

Community  Health  Centre  projects  indicate  that  there  may  be  further  merit 
in  comparing  emergency  and  urgent  care  approaches.  The  establishment 
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of  Emergency  Services  within  a primary  health  care  setting  is  a unique 
aspect  of  the  Northeast  Community  Health  Centre.  There  is  potential  to 
compare  types  of  clients,  presenting  needs,  and  outcomes  with  those  at 
an  “urgent  care”  centre  such  as  the  8th  and  8th  Health  Centre.  However, 
the  provision  of  a successful  free  standing  urgent  care  centre  as  an 
alternative  to  the  emergency  department  may  have  implications  for  service 
provision  under  the  Alberta  Health  Professions  Act,  which  requires  each 
health  profession  to  specify  which  of  its  members  are  regulated  to  perform 
a series  of  restricted  activities. 

Prevention/Promotion 

The  determinants  of  health,  health  promotion  and  injury/disease 
prevention  are  concepts  embedded  within  the  definition  of  primary  health 
care.  Evidence  from  the  Alberta  projects  supports  these  concepts  and 
emphasizes  the  importance  and  benefit  of  early  intervention  and  public 
awareness  strategies.  The  long-term  benefits  of  early  intervention  are  well 
documented.  Alberta  projects  such  as  Healthy  Families,  C.O.P.E.,  and 
Immunization,  serve  to  support  much  of  what  is  already  known  about  early 
intervention.  They  emphasize  that  primary  health  care  principles  applied  to 
early  intervention  strategies  for  child  health  are  particularly  effective. 

These  projects  illustrate  that  to  facilitate  access  to  primary  health  care 
service  (as  first  point  of  contact  with  the  health  system)  strategies  need  to 
be  developed  to  reach  populations  as  early  as  possible. 

These  findings  have  implications  beyond  children,  however.  Palliative  care 
projects  and  various  education  projects  (e.g.,  Misericordia,  Diabetes) 
indicate  that  early  intervention  strategies  are  effective  in  reaching  adults 
as  well.  The  concern,  though,  is  often  one  of  measurable  impact. 
Something  instituted  “early”  that  may  only  have  an  effect  “later”  can  try  the 
patience  of  systems,  organizations,  and  individuals  that  look  for  early 
returns  on  investment.  Many  of  the  project  evaluations  suggested  it  may 
be  as  long  as  5 to  10  years  before  the  full  impact  of  the  intervention  is 
realized.  This  has  significant  implication  for  the  roles  of  monitoring  and 
evaluation  in  primary  health  care  settings. 


Rural  Health 

It  is  difficult  to  discuss  primary  health  care  without  making  special  mention 
of  rural  Alberta.  The  Alberta  Primary  Health  Care  projects  which  were 
directed  at  better  understanding  primary  health  care  in  rural  settings 
tended  to  focus  on  issues  of  access.  For  example,  learning  from  the 
Health  Authority  5 project  demonstrated  that  introducing  multi-disciplinary 
teams  to  rural  communities  can  change  utilization  patterns  (i.e.,  fewer 
people  in  the  pilot  communities  reported  traveling  outside  the  region  to 
access  primary  health  services).  Telehealth,  while  in  its  infancy,  appears 
to  be  a viable  alternative  to  clients  having  to  travel  out  of  their  communities 
to  access  health  care  services,  and  to  alleviating  the  difficulty  of  drawing 
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physicians  to  northern  communities.  The  Settlement  Nurse  appears  to  be 
key  to  providing  primary  health  care  service  to  people  on  Metis 
Settlements.  The  Immunization  project  emphasized  the  importance  of 
personal  contact  when  health  services  are  not  within  easy  reach. 
Reminders  to  adhere  to  immunization  schedules  worked  successfully  to 
encourage  compliance.  Clearly,  different  strategies  may  be  required  to 
advance  primary  health  care  in  rural  Alberta  than  what  appears  to  have 
been  successful  in  projects  located  in  urban  centres. 

Support  for  Policy  Change:  Change  Management 

A major  challenge  in  advancing  primary  health  care  in  Alberta  is  change 
management.  Change  management  involves  the  development  of  new 
perspectives  and  practices  through  formal  and  informal  opportunities  to 
learn  how  to  do  things  differently,  and  what  benefits  arise  from  doing 
things  differently. 

Learning  to  Change 

The  C.O.P.E.  project,  for  example,  demonstrated  that  in  providing 
diagnostic  services  in  schools,  the  school  itself  can  serve  as  a training  site 
for  physicians,  psychiatry  residents,  teachers  and  Family  Liaisons.  The 
Multidisciplinary  Teams  project  illustrated  that  formal  workshops  and 
internet-supported  conferencing  tools  can  facilitate  consensus  building  in 
order  to  develop  guidelines  derived  from  practice-based  evidence.  These 
and  other  projects  illustrated  that  “real”  change  is  realized  at  the  front-lines 
where  services  are  delivered. 

Celebrating  Change 

It  is  also  important  to  recognize  that  change  occurs  at  different  rates  for 
different  sectors,  practitioners,  and  stakeholders.  For  example,  it  became 
readily  apparent  that  some  practitioners  have  a greater  proclivity  to 
working  in  team  environments  than  others.  Advancing  primary  health  care 
may  depend  on  “celebrating”  exemplars.  Another  critical  issue  is  to  realize 
that  changes  in  practice  should  not  be  instituted  just  for  the  sake  of 
collaboration  and  integration.  Not  all  circumstances  warrant  the  same 
approach  as  illustrated  by  examples  where  initiatives  in  the  Alberta 
projects  did  not  achieve  their  objectives — patients  with  chronic  pain  and/or 
fibromyalgia  did  not  respond  well  to  the  treatments  offered  within  the 
shared  care  framework. 
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Public  Awareness  Building 

But  to  advance  primary  health  care,  change  must  not  only  take  place  at 
the  point  of  service  delivery,  it  needs  to  be  understood  and  appreciated  at 
the  client  level.  Patients  must  recognize  the  merits  of  continuity  of  care 
over  the  “quick  fix”  provided  through  episodic  care.  For  example, 
prevention  and  promotion  require  extensive  public  awareness  building 
before  they  impact  significant  portions  of  the  population. 

Physician  Engagement 

There  was  significant  physician  engagement  in  the  Alberta  Primary  Health 
Care  project.  Many  physicians,  as  well  as  other  health  professionals, 
volunteered  significant  time  over  and  above  regular  duties  to  ensure  that 
projects  would  have  an  opportunity  to  succeed.  Sustained  involvement  of 
physicians  without  remuneration  cannot  be  expected.  However,  advancing 
primary  health  care  will  require  physician  engagement  of  the  type  and 
nature  witnessed  across  projects.  Not  only  will  motivational  strategies  be 
required  to  engage  physicians,  more  physicians  will  have  to  “see”  primary 
health  care  as  a viable  and  preferred  alternative  to  conventional  practice. 


Support  for  Policy  Change:  Regional/Community  Relationship 

Advancing  primary  health  care  in  Alberta  may  depend  on  the  ability  of 
regions  to  establish  close  working  relationships  with  their  communities  and 
for  communities  to  instill  a responsibility  for  health  in  individual  citizens. 
The  Airdrie  project  raised  specific  issues  of  regional  and  community 
relationship,  but  the  issues  may  be  illustrative  of  regional  and  community 
tensions  in  many  settings.  Through  the  evaluation  of  the  Airdrie  Regional 
Health  Centre  a number  of  suggestions  were  offered  as  reasons  why 
implementing  primary  health  care  using  a “health  centre”  approach  may  be 
more  difficult  within  a regionalized  system  than  as  a stand-alone  entity. 
Reasons  included: 

• the  need  for  Centres  within  a regionalized  system  to  negotiate 
community-based  vision  and  direction  with  competing  interests  and 
expectations  of  the  region; 

• the  influence  of  competing  demands  in  the  region  on  its  ability  to 
respond  to  the  needs  and  requests  of  the  health  centre; 

• lack  of  congruence  between  historical  organizational  structures  along 
functional  program  or  discipline  lines  with  the  primary  health 
care/community  health  approach  to  population  or  community-based 
planning; 

• tension  between  community  involvement  and  accountability  and 
pursual  of  partnerships  with  independent  health  service  providers. 
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Community  Action 


It  appears  that  community  coalitions  can  be  a successful  vehicle  for 
community-based  decision  making  and  action.  Coalitions  have  the 
opportunity  to  either  create  their  own  supportive  structure  or  use  an 
existing  structure  such  as  municipal  government  or  regional  health 
authority  (as  in  the  case  of  the  Healthy  Okotoks  Coalition).  This  may  be  a 
viable  mechanism  to  develop  and  support  relationship  between  regions 
and  communities.  Coalitions,  however,  require  significant  resource 
commitments.  The  Okotoks  project  demonstrated  that  after  one-and-one- 
half  years  and  31  planning  products,  15  community  actions  had  been 
achieved  to  effect  a single  change  in  the  community.  For  the  Healthy 
Okotoks  Coalition,  specifically,  progress  has  been  made  in  following  the 
fundamental  values  and  principles  of  health  promotion,  empowerment, 
and  community  development.  Further  work  is  needed  to  extend  the  work 
of  the  Coalition  into  the  broader  community  to  the  degree  the  membership 
feels  is  desirable  and  necessary  to  be  representative  of  community  health 
need.  Such  a move  would  require  membership  on  the  Coalition  of 
disenfranchised  community  members  as  well  as  ethnic  minorities. 

The  Metis  Community  Health  Council  appears  to  be  a good  mechanism 
for  Settlements  and  health  regions  to  learn  about  each  others’  cultures 
and  barriers  to  working  together.  The  combined  role  of  the  Settlement 
Nurse  and  Public  Health  Nurse  speaks  to  the  benefit  of  integrating  these 
roles  on  Metis  Settlements  to  provide  primary  health  care  services  in  an 
efficient  and  effective  manner.25 


It  was  a real  turning  point  when  the  realization  dawned  that  we  would  have  a say 
[in  selecting  the  Settlement  Nurse].  The  hiring  process  was  critical  in  building 
trust.  Lakeland  learned,  and  the  Settlements  started  to  understand  the  culture  of 
the  RHA  (Metis  Steering  Committee  member).  


Support  for  Policy  Change:  Monitoring  and  Evaluation 

Recurrent  themes  such  as  lack  of  baseline  data,  difficulty  in  discerning 
impact,  and  inadequate  measures  of  access,  quality,  and  integration 
illustrate  that  monitoring  and  evaluation  is  an  important  strategy  in 
advancing  primary  health  care.  The  following  represent  some  of  the  key 
implications. 

Primary  Health  Care  Evaluation  Framework 

Comprehensive  evaluation  requires  planning.  Developing  a provincial  level 
perspective  on  primary  health  care  may  benefit  from  a common  evaluation 
framework.  For  example,  an  evaluation  framework  for  community  health 
centres  would  facilitate  comparative  work  on  such  key  items  as 
governance  and  delivery  models. 


25  There  are  union  implications  with  this  approach. 
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Standardized  Data  Sets 


There  are  special  groups  for  whom  data  collection  (particularly  baseline 
data)  is  essential  if  the  impact  of  primary  health  services  is  to  be 
determined.  For  example,  there  is  need  to  identify  health  records  in  an 
appropriate  way  for  Metis  Settlements  people.  Regional  systems  of  data 
collection  (some  incompatible  with  each  other)  have  resulted  in  the 
decreased  ability  to  assess  certain  health-related  elements  on  a provincial 
basis — i.e.,  immunization  rates.  It  is  important  to  rebuild  capacity  to  assess 
advances  in  primary  health  care  through  common  data  collection  and 
information  sharing  protocols. 

Importance  of  Formative  Evaluation 

Formative  evaluations  have  provided  extensive  guidance  in  assisting 
programs  in  achieving  valuable  learning. 


Evaluation  was  introduced  at  the  beginning  of  operations  of  the  Centre;  an 
evaluation  “culture”  emerged  over  the  course  of  the  project  (NECHC  Independent 
Evaluation  Report). 


Also,  formative  evaluations  are  useful  in  identifying  issues  of  resource 
management.  For  example,  the  Brooks  Cares  project  demonstrated  that 
participatory  action  research  is  a strategy  that  has  the  potential  to  build  the 
skills  of  community  members  to  identify  and  address  issues  related  to 
health  and  well  being  of  community  members.  Communities  must  be 
socially  and  politically  prepared  to  accept  the  process  which  may  take 
three  to  five  years  to  implement  successfully. 

Another  resource  issue  emerged  through  project  experiences  with 
retrospective  evaluation.  Retrospective  evaluations  may  be  challenged  by 
staff  changes  and  clients  who  are  difficult  to  reach  (if  they  can  be  reached 
at  all).  In  some  cases,  people  are  asked  to  rely  on  long-term  memory  to 
recall  their  experiences  of  programs/services.  The  integrity  of  the  data, 
therefore,  may  be  questionable.  Only  when  retrospective  data  is 
necessary  for  comparison  purposes  should  retrospective  evaluations  be 
considered  for  funding. 

Evaluation  can  also  be  a motivator.  Linkages  between  research/evaluation 
and  improved  client  outcomes  can  serve  to  encourage  involvement  of 
health  professionals,  particularly  physicians,  in  data  collection  and 
recruitment  of  patients.  For  example,  once  oncologists  understood  that  a 
standardized  communication  template  was  appreciated  by  family 
physicians  and  ultimately  benefited  the  patient,  they  were  more  willing  to 
be  involved  in  providing  feedback  for  its  refinement. 
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Methodology 

Even  though  35  different  evaluators  participated  in  the  evaluation  of  the 
Alberta  Primary  Health  Care  project,  the  addition  of  a research/evaluation 
methodologist  in  the  early  stages  of  project  design  was  important. 
Increased  efficient  and  effective  processes  resulted  in  drawing  and 
learning  as  much  from  projects  as  possible.  These  types  of  experts  can 
provide  valuable  assistance  in  research/evaluation  design  (sampling 
procedures,  sample  size,  selection  of  control  groups,  etc.). 

Ethics  Review 

Primary  health  care  inquiries  often  necessitate  the  study  of  human 
subjects  and  they  require  some  form  of  ethics  review.  The  Alberta  projects 
experienced  delays  obtaining  ethics  approval  because  conventional  ethics 
approval  processes  are  typically  structured  around  research  and  provided 
through  academic  institutions.  To  facilitate  efficient  evaluation,  a modified 
version  of  the  standard  procedure  of  ethics  review  would  be  helpful. 

Further  Research 

Several  elements  of  primary  health  care  would  benefit  from  further  inquiry. 
This  implies  that  a research  agenda  directed  at  advancing  primary  health 
care  may  be  useful.  For  instance,  the  Shared  Care  project  illustrated  that 
high  levels  of  patient  satisfaction  are  not  necessarily  predictive  of  high 
levels  of  patient  perceived  health  status.  This  reinforces  the  complex 
(nonlinear)  relationship  between  access  to  health  services,  satisfaction 
with  services,  and  health  outcomes.  Other  examples  of  further  possible 
study  include 

• the  role  of  the  Nurse  Practitioner  in  hospital  and  other  settings, 
particularly  the  influence  and  impact  on  introducing  this  health 
professional  to  physician  environments; 

• different  types  of  reimbursement  (contract  and  fee-for-service)  and 
their  influence  on  scopes  of  practice  and  incentive  to  participate  in 
collaborative  activities; 

• the  impact  of  the  C.O.P.E.  intervention  on  improved  academic 
performance  of  C.O.P.E.  children,  follow-through  to  treatment, 
practice  variability  among  providers,  utilization  patterns  of 
emergency  and  acute  care; 

• the  best  mix  of  patients,  physicians,  mental  health  professionals, 
and  health  systems  that  produce  the  best  patient  outcomes; 

• longitudinal  studies  using  multiple  indicators  of  behaviour  change, 
investigation  of  follow  up  and  support,  comparison  of  different 
delivery  models,  barriers  to  attending  diabetes  centres;  and 

• The  impact  of  multi-disciplinary  teams  in  rural  settings. 
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Summary 


“I  would  have  lost 
my  son  that  day,  if 
it  hadn't  been  for 
Healthy  Families. 

I don't  know  of 
anything  else  that 
is  remotely  like  it 
anywhere  else. " 
(Healthy  Families 
Client) 


As  a result  of  receiving  $1 1 million  through  the  federal  Health  Transition 
Fund  (HTF),  Alberta  Health  and  Wellness  was  provided  with  the 
opportunity  to  study  and  evaluate  innovative  ways  of  delivering  health 
services  and  improving  the  health  system  in  Alberta.  In  consultation  and 
planning  sessions,  a number  of  regional  health  authorities,  communities, 
practices,  and  professional  organizations,  as  well  as  consultation  with  key 
stakeholders  indicated  support  for  the  enhancement  of  a primary  health 
care  approach26  for  both  rural  and  urban  areas  of  the  province.  Primary 
health  care  was  also  identified  as  a key  strategy  in  Alberta  Health  and 
Wellness’  Business  Plan  (1997-1998  to  1999-2000). 

A key  stakeholder  consultation  process  was  conducted  in  the  province  to 
inform  the  Call  for  Proposals  and  selection  of  projects.  This  resulted  in  a 
Draft  Discussion  Paper  on  Primary  Health  Care  which  presented  the  views 
of  multiple  stakeholders  on  key  components  that  should  be  considered  in 
the  development  of  future  policy  on  primary  health  care  in  Alberta. 


The  Alberta  Primary  Health  Care  project  was  then  formally  initiated  by  a 
provincial  Call  for  Proposals  in  April  1998.  An  external  Selection  Advisory 
Committee  reviewed  proposals  and  recommended  to  the  provincial 
Minister  of  Health  that  24  proposals  be  funded.  An  additional  three 
proposals  were  later  approved. 

A key  goal  of  the  Health  Transition  Fund  was  evaluation  of  projects  to 
identify  key  learning  in  priority  areas.  All  projects  were  expected  to  have 
measurable  outcomes  and  undergo  rigorous  investigation  of  project- 
specific  evaluation  questions  and  six  areas  of  inquiry  identified  at  the 
national  level.  The  six  dimensions  included  access,  quality,  impact  on 
populations,  integration,  cost-effectiveness,  and  transferability.  An 
Evaluation  Management  Team  (Howard  Research,  assisted  by  Wynne 
Resources)  was  contracted  to  provide  evaluation  support  to  projects, 
monitor  evaluation  activity,  and  synthesize  learning  across  projects. 


Learning 

A variety  of  approaches  was  used  to  advance  primary  health  care  in 
Alberta  including 


• rural/remote  access; 

• prevention  and  promotion  (community  development,  early 
intervention  and  education); 


Primary  health  care  is  considered  the  first  level  of  contact  of  individuals  with  the  health  care 
system — that  is,  where  health  services  are  mobilized  and  coordinated  to  promote  health,  prevent 
illness,  care  for  common  illness  and  manage  ongoing  health  problems  (National  Forum  on  Health, 
1997). 
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• integrated  service  delivery; 

• quality  improvement; 

• community  health  centre  models;  and 

• system  restructuring. 

Implementation  and  evaluation  of  these  approaches  have  resulted  in 
implications  for  advancing  primary  health  care  in  this  province. 

Implications  are  presented  below  followed  by  specific  contributions  to  each 
of  the  six  national  dimensions. 

Implications  for  Advancing  Primary  Health  Care  in  Alberta 


Integrated  Service  Delivery 


“Physicians  are 
able  to  spend  more 
time  with  needier 
patents.  Service 
has  been  able  to 
change  from  band- 
aid  to  more 
comprehensive, 
individualized  care 
that  has  made  a big 
difference  in  the 
lives  of  patients.” 
(Key  Informant, 
CUPS  Project- 
Calgary  Urban 
Project  Society) 


A great  many  projects  exemplified  integration  through  collaborative 
planning  processes  that  involved  consultation  with  stakeholders,  health 
professionals,  inter-agency  and  cross-sectoral  partnerships,  and  some 
negotiation  between  professional  organizations.  Successful  integration 
depended  on  the  willingness  of  health  professionals  to  cooperate — as  one 
evaluator  suggested,  moving  outside  “zones  of  comfort.”  Trust  was 
considered  central  to  relationship  development  and  team  building. 
Leadership,  often  by  physicians,  but  also  by  a variety  of  project  directors 
and  other  health  professionals  was  key  in  catalyzing  the  participation  of 
others.  Health  professionals  were  able  to  “see”  the  contributions  each  was 
making  to  the  care  of  clients  and  how  working  together  augmented  each 
other’s  contributions.  Professionals  became  aware  of  the  importance  of 
the  client  in  decision-making  processes.  Continuity  of  care  for  patients 
requires  an  integrated  system  of  service  delivery. 

Information  Sharing 


Information  sharing  among  stakeholders — providers,  clients,  public, 
agencies,  regional  administration  and  provincial  administration  is  key  to 
successful  integration.  There  appears  to  be  a need  for  clinically-integrated 
databases  where  on-line  real  time  information  is  simultaneously  available 
to  care  providers.  Such  information  sharing  would  support  inter- 
disciplinary collaboration  (e.g.,  through  electronic  real-time  health 
records).  Structures  and  processes  are  required  to  support  the  creation 
and  ease  of  use  of  such  databases.  Information  sharing  would  also 
facilitate  the  development  of  standardized  communication  mechanisms 
(templates,  electronic  health  records). 
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“We  have  a real  live 
example  of 
improving  care  in  the 
community  in  a team 
environment. . .not  for 
every  patient  or 
provider,  but  pretty 
good  evidence  for  its 
success  in  the  health 
system.  The  fact  that 
the  participants  are 
trying  to  think  of 
ways  to  continue  on 
an  ongoing  basis 
and  are  willing  to 
jump  over  more 
hurdles . . .obviously, 
they  want  it!” 

(Health  Care 
Provider,  Collectives 
Project) 


Methods  of  Payment 


Projects  emphasize  that  current  methods  of  payment  discourage 
involvement  in  collaborative  activity.  Differential  reimbursement  needs  to 
be  addressed  (e.g.,  psychiatrists  can  bill  for  consults  without  the  patient 
being  present  while  family  physicians  cannot).  In  addition,  current  fee-for- 
service  models  do  not  recognize  the  non-billable  time  a physician  may 
spend  with  clients  in  a primary  health  care  setting  (e.g.,  telephone 
consultations,  and  holistic  care).  Appropriate  reimbursement  systems  for 
physicians,  including  incentives  to  participate  in  interdisciplinary  and 
multidisciplinary  activity,  need  to  be  developed  and  implemented. 

Urgent  Versus  Emergent  Care 

The  issue  of  urgent  versus  emergent  care  highlights  the  importance  of 
clarifying  the  impact  on  resources  of  some  “first  contact”  services. 
Understanding  clients’  perspectives  towards  certain  services  helps  to 
distinguish  emergent  from  and  urgent  care.  Project  findings  point  to  the 
need  for  triage  (primarily  telephone  triage)  with  assistance  in  multiple 
languages. 


Prevention  / Promotion 


“Having  her 
(Settlement 
Nurse)  there  is 
a reminder  to 
be  proactive 
with  my  health.  ” 
(Community 
Member,  Metis 
Settlements 
Project) 


Determinants  of  health,  health  promotion,  and  injury/disease  prevention, 
are  concepts  embedded  within  primary  health  care.  Evidence  from  the 
Alberta  projects  supports  these  concepts  and  emphasizes  the  importance 
and  benefit  of  early  intervention  and  public  awareness  strategies  (e.g., 
Immunization,  Healthy  Okotoks,  Brooks  Cares,  Elnora,  Healthy  Families, 
C.O.P.E.,  Misericordia  Lifestyle). 

Rural  Health 


It  is  difficult  to  discuss  primary  health  care  without  making  special  mention 
of  rural  Alberta.  Projects  which  focused  on  understanding  primary  health 
care  in  rural  settings  tended  to  focus  on  issues  of  access.  Remote  health 
teams,  Settlement  Nurses,  telehealth,  and  immunization  schedule 
monitoring  may  serve  as  different  or  additional  strategies  to  advance 
primary  health  care  when  compared  to  what  have  been  successfully  used 
in  urban  centres. 


Strategies  to  Support  Policy  Change 

Projects  revealed  that  a number  of  strategies  can  contribute  to  a 
supportive  policy  environment  for  advancing  primary  health  care.  They 
include  change  management,  regional/community  relationship,  and 
monitoring  and  evaluation. 


rA 
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“She  wanted  to 
be  pain  free  and 
die  at  home  and 
the  Palliative 
Care  Program 
helped  us 
achieve  that 
goal.  ” (Family 
Member, 
Lakeland 
Palliative  Care 
Project) 


“Evaluation  is  part 
of  what  we  do 
here  now.  ” 
(Northeast 
Community  Health 
Centre,  Health 
Leader) 


Access 

“This  has  been  a 
big  help  for  rural 
populations  far 
from  a hospital  or 
clinic.  ” “I  would 
use  this  service 
more  than  driving 
40  miles  to  the 
doctor’s  office  in 
Strathmore.  ” 
(Community 
Member,  Health 
Authority  5) 


Change  Management 

Change  management  is  about  actively  managing  change  rather  than 
reacting  to  circumstances  that  cause  change.  Strategies  for  change 
management  include  opportunities  to  learn,  to  celebrate  success,  build 
awareness,  and  engage  physicians.  Supporting  a primary  health  care 
environment  will  also  include  encouraging  and  managing  the  transition 
from  episodic  to  continuous  care. 

Reoional/Communitv  Relationship 

Advancing  primary  health  care  in  Alberta  may  depend  on  the  ability  of 
regions  to  establish  close  working  relationships  with  their  communities  and 
for  communities  to  instill  a responsibility  for  health  in  individual  citizens.  It 
appears  that  community  coalitions  can  be  a successful  vehicle  for 
community-based  decision  making  and  action.  Coalitions  have  the 
opportunity  to  either  create  their  own  supportive  structure  or  use  an 
existing  structure  such  as  municipal  government  or  regional  health 
authority. 

Monitoring  and  Evaluation 

Recurring  themes  such  as  lack  of  baseline  data,  difficulty  assessing 
impact,  inadequate  measures  of  access,  quality,  and  integration  illustrate 
that  monitoring  and  evaluation  is  an  important  strategy  in  advancing 
primary  health  care  and  an  important  support  for  policy  change.  Strategies 
that  may  serve  to  continue  this  advancement  are  centred  in  developing  a 
provincial  primary  health  care  evaluation  framework,  developing 
standardized  data  sets  (or  criteria  for  producing  these  sets),  increasing  the 
role  of  formative  evaluation,  insisting  on  methodological  rigour,  addressing 
ethics  review  issues,  and  advancing  a research  agenda  on  primary  health 
care. 

Alberta’s  Contributions  to  the  Six  National  Dimensions 

Projects  identified  several  approaches  to  improving  access  including 
strategies  to  address  barriers  to  health  service. 

Project  learning  emphasized  that  models  of  service  delivery  in  a primary 
health  care  environment  need  to  include  a comprehensive  assessment  of 
community  members’  health  needs.  Involvement  of  community  members 
in  the  assessment  process  is  critical.  This  includes  identification  of 
“necessary”  services  and  subsequent  planning  for  health  services  which 
may  require  negotiation  with  existing  health  system  decision-makers.  It  is 
also  important  to  recognize  that  the  changing  demographic  profile  of 
communities  results  in  changes  in  community  need. 
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“In  the  absence  of 
any  pre-existing 
model  of 
multidisciplinary 
teamwork,  the 
Working  Group 
members  showed 
extraordinary 
flexibility  and 
adaptability.  ” 
(Multidisciplinary 
Teams  Project 
Report) 


Integration 


Projects  demonstrated  that  alternative  approaches  to  service  delivery  can 
enhance  access  to  primary  health  care  service  at  the  community  level. 
Mobile  health  teams,  for  example,  appear  to  work  well  in  rural 
communities.  Clients  appear  to  be  satisfied  with  services  provided  by 
Nurse  Practitioners  in  both  rural  and  urban  settings.  Medication 
compliance  appears  to  be  improved  through  collaborative  consultation  by 
care  providers.  Immunization  rates  can  be  improved  through  making 
personal  (telephone)  contact  with  parents.  Similarly,  when  Home  Visitors 
establish  personal  relationships  with  at-risk  families,  families  feel  more 
capable  of  accessing  the  wide  range  of  services  available  to  them — 
services  they  may  not  have  accessed  independently  (e.g.,  immunization 
and  Well  Baby  clinics). 

Coordinated  and  integrated  delivery  of  programs  appears  to  improve 
clients’  access  to  primary  health  services — particularly  access  to  mental 
health  services  and  palliative  care  services.  Project  findings  indicate  that 
community-based  mental  health  services  successfully  improve  physicians’ 
capacity  to  diagnose  and  treat  mental  health  problems.  Time  for  diagnosis 
of  children’s  mental  health  disorders  was  reduced  from  approximately  two 
years  to  two  weeks.  Opportunities  for  mental  health  therapy  have  been 
provided  to  newcomers  to  Canada. 

Project  learning  also  indicates  that  service  utilization  can  be  improved 
through  publicity.  Providing  reminder  notices,  for  example,  resulted  in 
improved  immunization  rates  and  improved  use  of  mental  health  services. 
Awareness  of  existing  programs  and  services  appears  to  be  an  important 
influencer  of  public  perception  of  access  to  service.  Perception  of  need  for 
service  also  appears  to  be  influenced  by  concern  for  one’s  health  or  that  of 
a family  member.  For  example,  people  appear  to  access  service  at  the 
Emergency  Department  out  of  worry  and  perception  of  urgent  need,  rather 
than  proximity  to  the  Emergency  Department,  or  lack  of  consultation  with 
another  care  provider. 

Alberta’s  Contribution  to  Understanding  Integration  and  Primary  Health 
Care 


Integration  and  coordination  are  essential  elements  of  a primary  health 
system.  Because  primary  health  care  is  broadly  focused,  an  integrated 
system  of  health  services  encompasses  the  continuum  of  services — 
physical,  mental,  and  many  would  argue,  spiritual  health  services  as  well. 
In  addition,  integrated  services  include  those  supporting  well-being  and 
quality  of  life  through  prevention  and  promotion  strategies.  Further, 
research  suggests  that  integration  has  the  potential  to  improve  health 
status,  user  satisfaction,  cost-effectiveness,  and  productivity. 

Several  of  the  27  projects  have  provided  examples  of  integration.  A key 
learning  to  achieving  integration  is  that  information  gathering  and 
information  sharing  are  critical  in  a primary  health  care  setting.  Without 
information  it  is  difficult  to  coordinate  services  and  assess  impact  of 
services  or  program  effectiveness  on  clients.  Project  findings  indicate  that 
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“My  ability  to  come 
up  with  the  most 
integrated  solution 
possible  was  done 
with  the 

observations  and 
expertise  of  other 
professionals  at  my 
disposal.  It  was 
easier  to  make  a 
decision  when  the 
group  made  a 
decision.  ” 
(Physician, 
Collectives  Project) 


Quality 


compatible  and  inter-operable  information  systems  and  other  tools  that 
facilitate  both  vertical  and  horizontal  integration  are  required.  Patients  and 
patients’  families  want  to  be  confident  that  their  health  service  provider 
(often  the  family  physician)  has  access  to  the  right  information  at  the  right 
time  that  will  help  to  determine  an  appropriate  course  of  action  and 
appropriate  treatment.  Both  urban  and  rural  palliative  care  projects 
emphasized  provider  and  patient  need  for  accurate  and  timely  information 
about  patient  care. 

Strong  leadership  and  appropriate  structures  and  processes  are 
necessary  to  support  an  integrated  health  system.  Integration  forces 
change.  Therefore,  change  management  strategies  are  needed  to  support 
integration  efforts. 

Integration  is  often  attempted  through  the  formation  of  interdisciplinary  and 
multidisciplinary  teams  of  health  providers.  Relationships  between  team 
members  are  important  and  can  be  facilitated  through  clear  definition  of 
roles  and  responsibilities.  Involvement  of  front-line  providers  is  essential 
since  change  needs  to  occur  not  just  in  theory  but  in  practice.  Establishing 
linkages  among  various  stakeholder  groups  (patients,  health 
professionals,  system  administrators,  public)  helps  develop  support  for 
new  approaches  to  integrating  services.  In  addition,  project  learning 
indicates  that  successful  efforts  to  integrate  services  respect  the  autonomy 
of  health  professionals  within  a collegial  environment  where  professionals 
are  encouraged  to  seek  advice  and  guidance  from  others. 

Project  findings  strongly  indicate  that  efforts  to  integrate  services  result  in 
improved  levels  of  satisfaction  among  providers  with  the  quality  of  care 
they  are  able  to  provide.  While  the  two-year  window  of  the  Health 
Transition  Fund  did  not  allow  for  definitive  conclusions  to  be  drawn, 
indications  point  toward  improved  patient  outcomes  as  well  (e.g.,  improved 
medication  compliance,  improved  diagnosis  of  children’s  mental  health 
disorders,  improved  diagnosis  and  treatment  of  adult  mental  health 
problems,  practice-based  clinical  practice  guidelines). 

Strategies  such  as  shared  client  records  and  communication  templates 
are  required  to  manage  changed  ways  of  interacting,  communicating,  and 
reporting  among  health  professionals  and  health  providers. 


Alberta’s  Contribution  to  Understanding  Quality  and  Primary  Health  Care 

Perception  is  an  important  determining  factor  in  assessment  of  quality. 
Users  of  health  services  often  link  quality  to  access  to  health  services.  For 
example,  people  who  receive  services  perceive  a positive  or  quality 
relationship  with  their  health  provider  as  a relation  of  the  amount  of  time 
the  provider  is  able  to  spend  with  them.  Projects  have  emphasized  that 
both  service  recipient  and  provider  would  like  to  spend  more  time  with  one 
another — or  perhaps  more  accurately,  appropriate  time  with  another.  The 
time  factor  appears  to  be  a significant  tension  in  the  provider/user 
relationship. 
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“The  doctors  are 
better  here 
because  they 
asked  more 
questions  when  I 
requested  a 
prescription.  ’’ 
(Service 

Recipient,  8th  and 
Health  Centre) 


Project  findings  indicate  that  providers  tend  to  take  a more  technical  view 
of  assessing  quality.  They  relate  quality  to  prescribed  standards  set 
through  clinical  practice  guidelines,  identified  and  measurable  indicators  of 
quality,  quality  improvement  processes,  chart  auditing,  and  so  on.  As  with 
the  integration  dimension,  information  sharing  among  providers  is  key  to 
providers’  perception  of  quality  service  delivery.  Providers  also 
acknowledge  that  the  quality  of  services  can  be  improved  through 
broadening  their  scopes  of  practice  (using  the  Nurse  Practitioner,  for 
example),  collaborating  with  other  health  professionals  (e.g.,  C.O.P.E., 
Shared  Care  Mental  Health),  and  clarifying  roles  among  providers 
(Collectives,  Health  for  All).  Findings  suggest  that  standardized 
communication  templates  used  between  oncologists  and  family  physicians 
enhances  the  quality  of  care  they  provide.  Further,  development  of  care 
plans  by  multidisciplinary  teams  for  chronic  illnesses  (e.g.,  diabetes, 
degenerative  joint  disease)  results  in  care  plans  that  are  applied  in 
practice  by  a variety  of  health  professionals  involved  in  the  care  of  patients 
(e.g.,  physician,  dietician,  therapist). 

One  of  the  challenges  to  assessing  quality  is  standardization  of  measures. 
Projects  have  found  that  while  some  standardized  instruments  are 
available  to  assess  quality  of  care,  modification  is  required  to  reflect  the 
specific  environment  in  which  care  is  provided.  Qualitative  indicators  are 
important  to  assess  the  “personal”  impact  of  services  on  clients.  One 
project  has  developed  a set  of  indicators  to  assess  the  quality  of  service  at 
a family  practice  medical  clinic.  Other  projects  used  mapping  exercises, 
model  development,  community  consultation,  and  continuous  quality 
improvement  processes  to  assess  the  quality  of  health  services  provided 
to  community  members.  These  strategies  were  used,  for  example,  to 
identify  the  needs  of  community  members  accessing  services  through 
community  health  centers  (urban  and  rural).  Information  of  this  type  is 
important  to  decision-makers  for  strategic  planning  exercises. 


Projects  also  identified  barriers  and  difficulties  to  improving  the  quality  of 
primary  health  care  services  including: 

(a)  identifying  needs  of  the  under-served  and  marginalized, 

(b)  inability  to  aggregate  data  for  certain  populations  (e.g.,  Metis)  due 
to  issues  of  privacy  and  protection, 

(c)  time  constraints  of  physicians  and  other  busy  health  professionals, 

(d)  unwillingness  to  share  information  across  sectors, 

(e)  deficient  information  systems,  and 

(f)  sampling  techniques  (e.g.,  exclusion  of  specific  audiences  such  as 
mental  health  patients). 
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Alberta’s  Contribution  to  Understanding  Impact  on  Populations 


Impact  on 
Populations 


While  impact  on  populations  (health  status)  was  identified  as  a key 
dimension  of  the  Health  Transition  Fund,  most  projects  did  not  address 
changes  in  the  health  of  the  population  overall.  For  many,  the  two-year 
time  frame  was  insufficient  to  implement  an  intervention  that  would  result 
in  measurable  change  in  health  status.  In  many  cases,  lack  of  baseline 
data  exacerbated  efforts  to  measure  change. 


“Thirty-seven 
percent  of  the 
under-immunized 
group  made  a 
change  in  their 
immunization 
status  following 
intervention  one  - 
Permission  to 
Contact  Letter.  ” 
(Immunization 
Project  Report) 


Health  status  outcomes  provide  both  strategic  and  implementation 
markers  for  health  services.  A broad  health  determinants  of  health 
perspective  of  health  status  necessarily  includes  a broad  range  of 
measures  such  as  economic  (income,  employment)  and  social  (physical, 
cognitive,  and  social  development  rates  of  family  violence)  indicators. 
Strategies  aimed  at  improving  the  health  of  the  population  through 
integration  of  services  show  promise,  but  require  longitudinal  study  to 
assess  change  (e.g.,  physician/psychologist/psychiatrists;  rural  health 
services  teams;  nurses/physician/pharmacist).  Some  projects  found  that 
tracking  the  impact  of  changes  on  transient  and  hard-to-reach  clients  is 
very  difficult.  Many  clients  had  no  health  care  number,  no  home  address, 
and  often  used  only  their  first  names  or  an  alias. 

Projects  have  indicated,  however,  that  rapport  between  client  and  provider 
is  critical  in  building  client  confidence  to  engage  in  behaviours  that 
improve  health.  However,  since  it  appears  that  some  groups  tend  to  be 
dependent  on  support  from  health  care  providers  to  maintain  their  health, 
a capacity-building  model  that  promotes  individual  responsibility  for  health 
will  need  to  be  used  to  facilitate  behaviour  change.  Community 
development  measurements  and  assessment  approaches  may  offer  some 
mechanism  to  link  strategies  targeted  at  improving  the  health  status  of  the 
individual  and  positively  impacting  the  health  of  the  population  overall. 


Cost- 

Effectiveness 


“I  think  this  is 
well  spent 
health  care 
dollars.  ” 
(Community 
Member, 
Health 
Authority  5 
Project) 


Alberta’s  Contribution  to  Understanding  Cost-Effectiveness  and  Primary 
Health  Care 


A complete  cost-effectiveness  assessment  examines  both  costs  and 
consequences  (psychological,  social,  behavioural,  and/or  physical  impact). 
In  addition,  a comparator  is  needed  to  determine  if  one  approach  has 
more  or  less  cost  and  more  or  less  effect,  relative  to  predetermined 
estimates  of  minimum  effect  and  maximum  cost. 

Only  one  of  the  Alberta  projects  was  able  to  meet  most  of  the  criteria 
required  for  a cost-effectiveness  analysis.  However,  several  projects  offer 
insight  into  cost-effectiveness  when  the  term  is  applied  to  a broader 
meaning  of  economic  analysis  including  cost,  cost-minimization,  cost- 
effectiveness,  and  cost-benefit.  Three  projects  (Healthy  Okotoks, 
Misericordia  Lifestyle  Improvement,  and  Health  Authority  5)  piloted  cost- 
effectiveness  and  sustainability  assessment  tools.  Most  other  projects 
were  challenged  by  the  perceived  technical  difficulty  associated  with 
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conducting  cost-effectiveness  assessments,  lack  of  appropriate  data  sets, 
and  difficulty  accessing  data  within  the  time  frame  of  the  project. 

Alberta’s  Contribution  to  Transferability  and  Primary  Health  Care 

Transferability 

A variety  of  tools  and  processes  have  been  developed  across  projects 
(e.g.,  instruments  to  measure  client  satisfaction,  template  to 
document/communicate  diagnostic  and  care  plan  information  for  palliative 
cancer  patients).  Some  have  used  standardized  instruments;  other 
instruments  have  been  tailored  to  the  Alberta  context  and  are  being  further 
refined.  Readers  may  wish  to  contact  the  developers  of  these  tools  to 
assist  in  establishing  indicators  of  validity  and  reliability.  In  addition  a 
number  of  models  have  been  developed  and/or  adapted  to  reflect  the 
Alberta  context  (e.g.,  Healthy  Families  model,  Integrated  Service  Delivery 
Model  for  Palliative  Care,  Continuous  Quality  Improvement  Model).  In 
addition,  a series  of  learning  derived  from  the  implementation  process 
itself,  has  been  compiled  (e.g.,  the  value  of  stakeholder  participation).  A 
complete  listing  of  tools  and  models  and  process  learning  can  be  found  in 
the  Appendix. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Rural/Remote  Access 


Rural  Primary  Health  Care  System  Pilot  Project 

(Demonstration  Project) 


Project  Name 

Rural  Primary  Health  Care  System  Pilot  Project 

(Demonstration  Project) 

Project  Description 

The  HA5  Rural  Primary  Health  Care  demonstration  project  was  an 
attempt  to  meet  the  health  needs  of  rural  residents  by  providing  a mobile 
team  of  health  professionals  to  the  community  on  a weekly  or  semi- 
weekly basis.  The  project  began  with  a community  needs  assessment, 
outreach,  and  mobile  services  including  physician,  public  health  nursing, 
pharmacy,  laboratory,  respiratory  technology,  speech  language 
pathology,  occupational  therapy,  and  mental  health  services  in  two  of 
the  three  pods  (Youngstown/Cessford  and  Cremona/Water  Valley).  A 
third  pod  was  activated  in  Standard/Hussar.  The  three  pods  now 
represented  an  extension  of  mobile  service  provision  in  the  eastern, 
western,  and  southern  parts  of  Regional  Health  Authority  5.  A quasi- 
experimental  pretest/post-test  nonequivalent  comparison  group 
research  design  was  used  to  measure  rural  community  residents’  self- 
perceived  health  status,  health  awareness,  health  attitude  and 
behaviour,  and  perceptions  of  health  services  quality  and  access. 

Expected  Outcomes 

1 . Well-functioning  health  care  teams  to  provide  primary  health  care 
services  to  rural  communities. 

2.  Improved  access  to  primary  health  care  services  to  under  serviced 
rural  communities. 

3.  Improved  community  members'  perception  of  the  quality  of  primary 
health  care  services  provided  to  the  community. 

4.  Local  health  needs  met  based  on  assessment  data,  community 
concerns,  and  health  provider  input. 

5.  Linkages/referrals  with  appropriate  agencies  as  required. 

6.  Decreased  numbers  of  individuals/families  seeking  primary  health 
care  services  outside  the  region. 

7.  Improved  health  status  of  individuals  resident  in  remote/rural 
communities. 

8.  Issues  of  sustainability  identified  and  recommendations  provided. 

Methodology  of 

Independent 

Evaluation 

Process  Evaluation 

The  process  evaluation  was  exploratory  and  descriptive  in  nature.  Data 
collection  included  observation,  site/telephone  interviews,  tracking  of 
client  feedback,  document  review,  consultation  with  Steering 
Committee  and  project  coordinator,  unsolicited  feedback  from 
communities  and  staff,  interviews  with  physicians  who  were  part  of 
mobile  service  teams,  as  well  as  Executive  Director  and  Medical 
Director. 

Outcome  Evaluation 

A quasi-experimental  pretest/post-test  nonequivalent  comparison  group 
research  design  was  used  in  this  study.  A structured  telephone  survey 
(primarily  forced  choice  with  some  open-ended  questions)  was 
administered  to  600  community  members  at  Time  1 (project  initiation) 
and  Time  2 (project  close).  Of  the  total  600  interviews,  574  people 
responded  to  both  Time  1 and  Time  2 surveys  (13.5%  attrition  rate). 
The  matched  pair  response  rate  is  high  and  was  necessary  to  measure 
changes  in  the  variables  between  the  time  prior  to  the  intervention  of 
the  mobile  teams  visiting  the  communities  and  subsequent  to  their 
intervention  (a  period  of  12  months). 
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Review  of  Project 
Implementation 

Prior  to  introducing  mobile  services  to  the  communities,  the  project 
coordinator  conducted  extensive  community  awareness  activities — 
formally  and  informally  with  individuals,  groups,  and  community 
meetings.  Local  media  covered  the  process.  Links  were  made  to  the 
informal  network  of  community  leaders. 

The  first  two  pods  were  launched  in  January  1999,  the  third,  in  June 
1999.  Rural  pods  (communities)  received  mobile  services,  including 
physician,  public  health  nursing,  pharmacy,  laboratory,  respiratory 
technology,  speech  language  pathology,  occupational  therapy,  and 
mental  health  services.  Town  administration,  credit  unions,  schools, 
seniors’  coalitions,  women’s  groups  and  local  business  became 
involved  in  making  decisions  about  location  and  scheduling  of  services. 
Regional  Board  members,  management  and  staff  were  also  involved.  In 
consultation  with  the  community  the  project  coordinator  arranged  for 
the  approval  and  supply  of  required  equipment  and  payment  of 
physician  stipends. 

Measurement  of  community  self-perceived  health  status,  awareness  of 
local  services,  and  access  to  services  was  conducted  in  November 
1998,  through  a random  sample  of  community  members.  A follow-up 
survey  (with  the  same  random  sample  of  community  members)  was 
conducted  in  the  spring  of  2000. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Strong  coordination  and  communication  with  communities 

■ Cooperation  from  professional  providers 

■ Need  for  community-based  resources — good  uptake  (utilization)  by 
community  residence 

Challenges 

■ Short  timelines 

Results 

Process  Evaluation 

1.  Strong  and  positive  relationships  were  reported  among  members  of 
the  mobile  teams  and  between  staff  and  community  members  of 
the  teams.  There  was  a solid  sense  of  security  and  harmony 
among  professional  providers. 

2.  Good  liaison  with  senior  management  developed. 

3.  Treatment,  prevention,  and  health  promotion  services  were 
provided. 

4.  Service  expansion  included  links  to  diagnostic  services  (lab  and 
pharmacy). 

5.  The  number  of  referrals  increased,  and  new  referral  patterns 
emerged. 

6.  Numbers  of  patients  per  clinic  ranged  from  19  in  Cremona,  to  3.6 
per  clinic  in  Youngstown.  Young  children  and  seniors  were  the 
largest  groups  utilizing  the  services.  Four  main  diagnostic  areas 
dealt  with  respiratory,  circulatory,  nervous  system,  and  mental 
disorders. 

7.  According  to  the  client  surveys  for  clinic  services,  most  people  in 
Cremona,  Cessford,  and  Youngstown  were  first  time  clinic  goers. 
The  majority  in  Hussar  was  repeat  visits.  Satisfaction  ratings  were 
"very  satisfied"  and  "extremely  satisfied." 

8.  Expansion  of  services  to  other  disciplines  and  diagnostic  services 
was  based  on  community  assessment  of  need  or  interest. 

9.  Ethical  concerns  were  raised  by  staff  about  lack  of  sustainability 
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and  its  impact  on  community  relations. 

Outcome  Evaluation 

1 . There  is  unequivocal  evidence  that  treatment  communities  were 
aware  of  the  new  health  services  and  did  not  confuse  these 
services  with  other  health  or  health-related  services. 

2.  There  is  very  little  evidence  to  suggest  that  any  of  the  communities 
changed  their  self-perception  of  physical  or  mental  health. 

3.  While  not  conclusive,  evidence  suggests  that  multidisciplinary  rural 
health  services  and  community  development  activity  have  a 
positive  impact  on  self-perceived  health  knowledge. 

4.  One  community  appeared  to  perceive  that  the  quality  of  health 
services  in  the  community  had  improved. 

5.  Evidence  supported  the  contention  that  multidisciplinary,  rural 
health  services,  and  community  development  activities  can  have  a 
positive  impact  on  health  attitudes  and  behaviours. 

6.  Treatment  communities  reported  a significant  increase  in  complying 
with  prescribed  medication  regimens  and  flu  shots,  whereas  the 
control  community  did  not.  While  this  finding  can  be  interpreted  as 
increasing  cost  overall,  this  finding  also  points  to  increased  access 
to  health  services  and  improved  quality  of  service. 

7.  All  four  communities  reported  an  increase  in  use  of  the  internet  as  a 
means  to  locate  health-related  information. 

8.  Other  findings  of  regional  interest  emerged  including 

■ fewer  people  leaving  the  region  to  access  physicians’  services 
(especially  in  Calgary); 

■ increases  in  the  number  of  people  traveling  to  Calgary  to 
access  specialist  services; 

■ higher  proportion  of  people  in  the  region  seeking  health 
information  or  having  received  health  information; 

■ fewer  people  agreeing  that  drinking  alcohol  reduces  health 
status; 

■ general  decline  in  smoking; 

■ higher  proportion  of  people  trying  to  increase  the  amount  of 
fruit  and  vegetables  in  their  diets; 

■ increased  numbers  of  people  having  their  blood  pressure 
checked. 

Learning 

This  project  manifested  all  the  growing  pains  of  forming  new  groups  ! 

and  became  “the  little  project  that  could.”  For  the  purposes  of  sharing 

lessons  learned,  project  participants  attributed  the  endurance  and 

success  of  the  project  to 

■ the  excitement  about  the  vision  and  commitment  to  implementation 
of  the  mobile  team,  (especially  the  multidisciplinary  professionals) 
established,  lead,  and  nurtured  by  the  project  coordinator; 

■ continued  championing  of  the  vision  by  the  CEO; 

■ skills,  tenacity,  and  community  orientation  of  the  project 
coordinator; 

■ willingness  of  physicians  to  provide  services  in  the  communities; 

■ financial  reimbursement  of  physicians  for  their  travel  time  and 
services;  and 

■ membership  and  reporting  relationships  of  the  Steering/Advisory 
Committee  to  senior  administration. 
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Contribution  to  National  Dimensions 

Access 

■ All  three  communities/pods  embraced  the  provision  of  services, 
increased  availability  of  health  information,  and  the  community 
development  initiatives 

Quality 

■ While  some  statistical  evidence  suggested  that  services  provided  in  the 
communities  were  perceived  by  community  members  as  being  of  higher 
quality  pre  and  post-test,  community  users  of  the  services  consistently 
rated  the  services  received  as  "very  good"  or  "excellent.” 

Integration 

Health  Status 

Cost- 

Effectiveness 

■ A cursory  examination  of  the  cost-effectiveness  data  suggested 
that  gains  made  in  each  community  were  worth  achieving. 
However,  the  region  must  determine  whether  results  merit 
mobile  delivery  of  health  services  again. 

Transferability 

■ There  is  definite  potential  for  transferability.  Extensive  data  have  been 
collected  on  operation  and  cost. 

■ The  community  survey  instrument  used  requires  further  validity  and 
reliability  testing. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 


Policy 

Implications 

■ This  study  raises  the  viability  and  worth  of  providing  distributed  primary 
health  care  services  in  rural  areas. 

Implications  for 
Research  and 
Evaluation 

■ Indicators  of  primary  health  care  influences  are  required. 

■ Community  surveys  need  validation  and  reliability  assessments  based 
on  indicators  selected  to  measure  impact  of  primary  health  care 
initiatives.  In  the  meantime,  some  standardized  elements  of  health 
status,  quality  and  access,  need  to  be  introduced  to  community  surveys 
to  provide  comparison  over  time  and  across  populations. 

■ Some  indications  from  this  evaluation  suggest  that  rural  community 
primary  health  care  initiatives  may  be  cost-effective.  A more  detailed 
examination  of  cost-effectiveness  is  required. 

Synopsis  of  Independent  Evaluation  Reports 


Project  Name 

Health  For  All  Project  - Metis  Settlements  Project 

(Demonstration  Project) 

1 Project 
Description 

The  Metis  project  involved  the  addition  of  a Settlement  Nurse  to  four  Metis 
Settlements  (Elizabeth,  Fishing  Lake,  Buffalo  Lake,  and  Kikino)  located  in 
the  Lakeland  Regional  Health  Authority.  While  the  project  was  intended  to 
involve  Metis  community  members  in  identifying  health  needs,  identifying 
priorities  and  strategies  to  address  priorities,  and  to  facilitate  integration 
with  other  parts  of  the  health  system  and  other  health  stakeholders,  the 
primary  result  was  the  building  of  a trusting  relationship  between  the 
Settlements  and  the  region.  The  foundation  has  been  laid  to  continue 
working  together  to  address  health  needs.  The  Settlement  Nurse  and 
Public  Health  Nurse  developed  a new  way  of  coordinating  their  services  on 
the  Settlements. 

Expected 

Outcomes 

1 . Enhancement  of  the  multidisciplinary  team  providing  public  health  and 
home  care  services  on  the  four  Metis  Settlements  by  the  addition  of  a 
Settlement  Nurse. 

Methodology  of 

Independent 

Evaluation 

Document  review,  weekly  logs  from  Settlement  Nurses  and  Aboriginal 
Liaison  Worker,  with  monthly  roll-up  and  report,  face-to-face  and  telephone 
interviews  (individual  and  group),  community  survey  (primarily  closed 
questions),  and  direct  observation  Steering  Committee  and  Community 
Health  Council  meetings. 

Review  of  Project 

Implementation 

Process 

Initial  expected  project  outcomes  were  significantly  revised  approximately 
mid-way  through  the  project  (to  what  appears  above). 

Settlement  Nurses  were  hired  by  the  region  in  consultation  with 
representatives  from  the  Metis  Settlements.  Settlement  Nurses  (two  on  one 
Settlement  each,  one  Nurse  split  between  two  Settlements)  lived  on  the 
Settlements  and  provided  health  services  on  site. 

Facilitators/ 
Challenges  to 
Implementation 

The  role  of  the  Settlement  Nurse  revised  to  support  Metis  communities  to 
identify  strengths  and  available  resources;  support  Metis  communities  in 
identifying  the  main  determinants  of  health  which  impact  their  communities; 
increase  awareness  and  utilization  of  existing  health  programs  and 
services;  mobilize  community  members  to  have  a proactive  and  integral 
role  in  improving  the  health  of  their  community;  assess,  develop, 
implement,  and  evaluate  appropriate  community-based  programs  and 
services  to  maximize  community  health;  approach  Metis  clients  in  a 
culturally  sensitive  manner;  communicate  and  collaborate  with  co-workers, 
other  professionals,  and  community  agencies  to  provide  appropriate 
services;  act  as  a client/community  advocate;  maintain  complete  and 
accurate  records;  and  ensure  that  programs  and  services  were  safe, 
accessible,  acceptable,  appropriate,  efficient,  effective,  and  facilitated 
continuity  of  service. 

Roles  of  the  Aboriginal  Liaison  Worker  (who  acted  as  the  regional  liaison 
between  Metis  communities  and  community  organizations)  and  Community 
Health  Representative  who  worked  with  the  Public  Health  Nurse  and 
assisted  individuals,  families,  and  groups  on  the  Settlement,  were  also 
clarified. 

Facilitators 

■ Strong  commitment  of  senior  management  and  Metis  Community 
Health  Council  to  work  together. 

■ Second  round  of  advertising  for  Settlement  Nurses  was  successful  in 
securing  Nurses  with  appropriate  qualities  and  skill  sets. 

Health  For  All  Project  - Metis  Settlements  Project 

(Demonstration  Project) 
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■ Metis  representatives  were  involved  in  hiring  Settlement  Nurse 
(signified  shared  power). 

■ Collaborative  revisioning  of  project  outcome  (evaluators,  region, 
Community  Health  Representatives). 

■ Revised  role  of  Settlement  Nurses  and  clarified  roles  of  other  key 
stakeholders. 

■ Aboriginal  Liaison  Worker  helped  Settlement  Nurses  understand  the 
culture  and  personalities  on  each  Settlement. 

Challenges 

■ Organizational  restructuring  of  the  health  authority  occurred  during  the 
early  stages  of  the  project.  This  had  a negative  impact  on  staff  morale. 

■ There  was  a serious  underestimation  of  time  required  to  obtain 
information  to  substantiate  improvements  in  access  to  primary  care 
services:  lack  of  a mechanism  for  arranging  for  aggregate  data  about 
the  usage  of  hospital  and  emergency  services  by  Settlement  residents. 
Attempts  to  obtain  data  on  community  members'  use  of  primary  care 
physicians  from  Alberta  Health  and  Wellness  based  on  postal  code 
failed  because  community  members  refused  to  give  consent.  Computer 
problems  within  the  LRHA  resulted  in  data  from  the  Public  Health 
information  System  being  unavailable  for  analysis. 

■ Initially  there  was  a mismatch  in  the  community's  vision  of  a healing 
place/wellness  centre  and  the  part-time  Settlement  Nurse  proposed  by 
the  region.  The  Settlements  hoped  for  a focus  on  mental  health,  home 
care,  and  substance  abuse  rather  than  public  health. 

■ Initially,  there  was  a lack  of  communication  between  the  Public  Health 
Nurses  and  the  Settlement  Nurses. 

■ Insufficient  time  for  head  of  Public  Health  Services  to  supervise 
Settlement  Nurses  in  addition  to  her  other  duties. 

■ Location  of  Settlement  Nurses'  offices  in  non-high  traffic  areas  of  the 
Settlement  meant  access  was  not  optimal. 

■ Changes  in  the  Community  Health  Representative  and  Metis 
Community  Health  Council  meant  changing  membership  on  the 
Steering  Committee. 

■ Aboriginal  Liaison  Worker  indicates  community  politics. 

■ Community  Health  Council  provides  structure. 

Results 

■ While  the  Settlement  Council  was  aware  of  the  Settlement  Nurse  and 
her  role,  the  broader  community  was  unclear  about  the  new  services 
that  were  available  to  them. 

■ Generally,  trust  was  established  between  Settlement  Nurses  and 
community  members. 

■ Approximately  three  quarters  of  community  members  who  responded 
to  the  community  survey  reported  that  they  had  made  contact  with  the 
Settlement  Nurse. 

■ Approximately  one  third  of  survey  respondents  indicated  that  they  had 
worked  with  the  Settlement  Nurse  to  plan  a health  event. 

■ Settlement  Nurses  provided  transportation  to  events  (welcomed  by 
seniors). 

■ Settlement  Nurses  made  appointments  to  providers  outside 
Settlements. 

■ 80%  of  survey  respondents  indicated  that  the  most  significant  benefit  of 
the  Settlement  Nurse  was  that  she  provided  quick  answers  to  health 
questions  (saving  time  and  expense  of  travel). 

■ 60%  of  survey  respondents  indicated  that  the  Settlement  Nurse  had 
assisted  them  to  take  action  to  improve  their  health. 
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■ Steering  Committee  members  noted  that  community  members  were 
involved  in  planning  and  evaluating  the  project. 

■ Settlement  Nurse  represented  a significant  increase  in  access  to 
services. 

■ The  health  authority  and  the  Settlements  had  increased  capacity  to 
engage  in  a partnership  project  than  they  did  when  they  started. 

■ Settlements  were  committed  to  maintain  the  Nurses. 

■ Settlements  and  the  health  authority  experienced  an  increased 
awareness  of  each  other’s  culture. 

■ Steering  Committee  members  were  committed  to  the  project. 

Learning 

■ More  Community  Health  Council  members  believed  that  they  were 
making  a difference  and  that  the  health  authority  was  listening  to  them. 

■ The  project  improved  relationships  between  the  region  and  the  Metis 
Settlements. 

■ Regular  Steering  Committee  meetings  where  all  parties  came  together 
were  important. 

■ The  attitude  of  the  Public  Health  Nurses  changed  due  to  role 
clarification  of  the  Settlement  Nurse. 

■ It  would  be  helpful  if  the  Settlement  Nurse  spoke  Cree  (especially  for 
elderly  community  members). 

■ Community  members  continue  to  want  a treatment  clinic. 

■ Settlement  Nurses  feel  isolated  on  the  Settlement. 

■ The  Aboriginal  Liaison  Worker  was  helpful  in  assisting  Settlement 
Nurse  to  understand  the  culture  and  personalities  of  the  Settlement. 

Contribution  to  National  Dimensions 

Access 

■ The  Public  Health  Nurses  and  Home  Care  Nurses  did  not  see  any 

improvement  in  the  knowledge  of  community  residents  about  Public 
Health  and  Home  Care  Services  available  in  the  region.  However,  the 
community  members  who  responded  to  the  survey  commented 
frequently  on  the  ease  of  access  to  the  services  provided  on  the  i 

Settlement  by  the  Settlement  Nurses.  This  indicated  savings  in  time  and 
cost  of  travel  off  the  Settlement.  The  Nurse  shared  preventative  and 
health  promoting  information  with  community  members. 

■ Community  member  recognized  that  the  Settlement  Nurse  had 

information  and  could  link  them  to  available  services  if  they  required  j 

them. 

Quality 

■ Results  of  the  community  survey  indicated  that  respondents  were 
satisfied  with  the  concept  of  the  Settlement  Nurse,  although  there  was  no 
evidence  that  satisfaction  with  other  services  provided  by  the  region  has 
changed.  However,  relationships  between  Metis  representatives  and  the 
region  have  improved  due  to  their  involvement  with  the  region  in 
planning  and  developing  health  services. 

■ Community  members  had  come  to  understand  that  the  Nurse  could  help 
them  identify  health  issues  and  assist  with  treating  the  illness  at  an 
earlier  stage. 

■ Anecdotal  evidence  suggested  that  the  presence  of  the  Settlement 
Nurse  brought  health  to  the  forefront  and  supported  more  people  to 
action  to  improve  their  health. 

Integration 

Health  Status 

■ Anecdotal  evidence  pointed  to  improved  health  status  of  specific 
individuals  as  a direct  result  of  the  placement  of  a Nurse  on  the 
Settlements. 

Synopsis  of  Independent  Evaluation  Reports 


1 Cost- 

Effectiveness 

Transferability 

■ Description  of  a model  for  introduction  of  a Settlement  Nurse  to  a Metis 
Settlement. 

■ The  first  and  most  critical  strategy  to  increase  knowledge  and  awareness 
of  the  residents  was  to  gain  their  acceptance  and  trust. 

■ Convenient  access  increased  the  likelihood  that  a health  practitioner  will 
be  there  when  a community  member  wanted  to  become  involved  in 
identifying  and  developing  strategies. 

■ Having  a Nurse  in  the  community  working  with  individuals  to  identify 
needs  helped  to  identify  groups  with  common  health  needs. 

■ The  project  demonstrated  the  importance  of  willingness  on  the  part  of  the 
regional  executive  to  engage  directly  with  Settlement  leaders  in  a 
process  of  community  building,  not  just  delivering  services. 

■ Early  establishment  of  the  Aboriginal  Liaison  Worker  and  Community 
Health  Representatives  for  each  Settlement  helped  to  create  links  to  the 
Settlements. 

■ It  is  necessary  to  use  a community  needs  survey  to  identify  health  needs 
and  monitor  success  and  to  consult  community  on  introduction  of  the 
Settlement  Nurse  to  the  community.  The  Settlements  need  to  be 
involved  in  the  hiring  process  and  establishing  a location  for  the  Nurse 
on  the  Settlement.  Settlement  officials  need  to  be  involved  in  introducing 
the  Nurse  to  community  members. 

■ Effective  supervision  and  support  structures  for  the  Nurses  to  counteract 
isolation  are  required.  It  is  important  to  build  strong  connections  to  other 
regional  services. 

■ Provide  opportunities  for  regional  staff  and  officials  to  learn  about  Metis 
culture. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 


Policy 

Implications 

■ The  Metis  Community  Health  Council  appears  to  be  a good  mechanism 
for  Settlements  and  regions  to  learn  about  each  other’s  cultures  and 
barriers  to  working  together  (e.g.,  union  constraints). 

■ Settlement  Nurses  appear  to  increase  access  to  health  services  on  Metis 
Settlements,  but  as  representatives  of  the  region,  they  must  first  build 
relationship  and  trust  with  community  residents. 

■ The  Aboriginal  Liaison  Worker  and  Community  Health  Representative, 
as  well  as  regional  representatives  have  an  important  role  to  play  on  the 
Steering  Committee  to  plan,  organize,  and  implement  health  services  on 
Metis  Settlements.  The  combined  role  of  Settlement  Nurse  and  Public 
Health  Nurse  speaks  to  the  need  for  an  integrated  role  of  Settlement 
Nurses  on  Metis  Settlements. 

■ It  may  be  beneficial  for  Settlement  Nurses  to  speak  Cree,  particularly 
when  working  with  seniors. 

Implications  for 
Research  and 
Evaluation 

■ Need  for  comprehensive  baseline  on  community  members'  knowledge  of 
health  services,  utilization  rates,  cost  analysis  of  travel  to  health  services, 
etc. 

■ Need  to  clarify  data  requests  early  (of  the  region,  of  Alberta  Health  and 
Wellness). 

■ Need  to  identify  health  records  in  an  appropriate  way  for  Metis 
Settlements  people  (e.g.,  postal  code  does  not  seem  to  work  since 
community  members  collect  mail  from  various  locations). 

rA 


8 


Synopsis  of  Independent  Evaluation  Reports 


■ Need  for  enhanced  regional  data  collection  systems  (e.g.,  utilization  data 
for  hospitals  and  emergency  services). 

■ Need  to  triangulate  perspectives  (Community  Health  Representatives, 
Public  Health  Nurses,  and  Home  Care  Nurses). 

■ Need  much  stronger  evaluation  design  to  gather  baseline  data — even  on 
self-reported  health  status. 

■ Need  for  collaborative  identification  of  realistic  project  outcomes 
(evaluators  and  project  participants)  and  indicators  of  success. 
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Project  Name 

Telehealth 

(Demonstration  Project) 

Project 

Description 

This  demonstration  project  evaluated  the  effectiveness  of  Telehealth  in 
providing  enhanced  primary  health  care  services  to  three  northern, 
geographically  remote  communities  (Trout  Lake,  Peerless  Lake,  and  Red  Earth 
Creek)  in  Keeweetinok  Lakes  Regional  Health  Authority  (KLRHA).  The 
effectiveness  of  telehealth  in  providing  improved  health  care  services  to  these 
communities  was  examined. 

Expected 

Outcomes 

1 . Telehealth  will  increase  client/provider  satisfaction  with  quality  of  health 
services  delivered. 

2.  Customized  telehealth  programs  will  be  developed  to  meet  special  client 
needs. 

3.  Telehealth  will  result  in  increased  access  to  specific  health  services. 

4.  Telehealth  will  reduce  clients’  need  to  travel  to  outside  communities  for 
health  care  services. 

5.  Telehealth  will  facilitate  public/health  stakeholder  participation  in  health  care 
planning. 

6.  Telehealth  will  reduce  expenditures  for  individuals/families  and  for  the 
KLRHA  for  staff/employer  costs. 

7.  Telehealth  will  demonstrate  its  utility  in  remote  communities  for  (a)  providing 
quality  services,  (b)  increasing  access,  (c)  facilitating  public  participation  in 
planning,  and  (d)  reducing  personal  and  KLRHA  staff  expenditures 

Methodology  of 

Independent 

Evaluation 

■ Goals,  objectives,  and  outcomes  were  collaboratively  developed  with 
project  staff. 

■ Process  and  outcome  evaluation  measures  were  identified. 

■ Instruments  for  collecting  statistical  and  perceptual  data  including 
Telehealth  database,  Telehealth  session  logs,  Telehealth  evaluation  log, 
client  satisfaction  questionnaire,  and  interview  questions  for  key  informants 
were  developed. 

■ Regular  meetings  and  teleconferences  were  held  between  the  evaluator 
and  project  staff. 

Review  of  Project 

Implementation 

Process 

■ Prior  to  the  commencement  of  the  project,  the  KLRHA  had  purchased  and 
installed  telehealth  equipment  in  High  Prairie,  Slave  Lake, 
Wabasca/Demarais,  Trout  Lake,  Peerless  Lake,  and  Red  Earth  Creek.  This 
equipment  consisted  of  dedicated  computers,  video/audio  bridging 
equipment,  peripheral  devices  (e.g.  stethoscopes)  and  satellite  uplinks. 

■ Many  of  the  first  telehealth  logged  sessions  throughout  the  region  were  to 
“test”  the  system. 

■ The  telehealth  coordinator  essentially  spent  the  first  year  (1999)  providing 
technical  support  and  troubleshooting  services  to  ensure  that  telehealth 
sessions  throughout  the  KLRHA  region  were  delivered  as  smoothly  as 
possible.  The  coordinator  spent  much  of  his  time  implementing  a tele- 
ultrasound project  in  Wabasca/Demarais  and  in  facilitating  sessions 
developed/delivered  by  other  health  service  providers  (e.g.  UofA  geriatrics 
project).  It  wasn’t  until  after  January  2000  that  the  majority  of  telehealth 
sessions  were  delivered  in  the  communities  of  Red  Earth  Creek,  Peerless 
Lake,  and  Trout  Lake. 

■ As  a result,  42  telehealth  sessions  were  ultimately  delivered  in  these  three 
northern  communities. 

■ A computer  database  was  developed  and  this  database  was  ultimately 
expanded  to  record  all  telehealth  sessions  in  KLRHA  communities. 

■ To  measure  client/provider  satisfaction,  four  survey  questionnaires  were 
developed  and  one  included  questions  for  estimating  client  cost-savings 
realized  from  the  client  not  having  to  leave  the  community  for  health 
services.  However,  these  questionnaires  were  not  utilized  during  the 
project,  with  the  exception  being  that  two  speech  pathology  clients 
completed  client  satisfaction  questionnaires. 
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Facilitators/ 
Challenges  to 
Implementation 

Challenges 

■ Project  started  slowly  due  to  difficulty  in  recruiting  Telehealth  Coordinator 
(TC)  and  research  assistant  (the  latter  person  subsequently  resigned  and 
was  replaced). 

■ No  needs  assessment  process  was  in  place  to  advise  the  development  of 
Telehealth  applications. 

■ From  the  onset  of  the  project  there  were  many  barriers  impeding  the 
outcome  of  this  project: 

■ The  telehealth  equipment  and  associated  technology  was  not  up  to  par 
and  ready  to  facilitate  the  project; 

■ There  were  numerous  problems  with  the  satellite  technology; 

■ Staffing  resources  were  not  in  place  at  the  beginning  of  project; 

■ When  staffing  was  in  place,  they  were  constantly  having  to  deal  with 

ambiguity  of  their  roles  within  the  scope  of  the  project;  ! 

■ Project  team  members,  providers,  and  clients  were  sufficiently  trained; 
and 

■ TC  spent  most  of  his  time  troubleshooting/upgrading/adjusting  the 

technology  and  mounting  a separate,  related  project — tele-ultrasound  in 
Wabasca/Demarais.  ! 

Results 

■ Because  no  baseline  data  were  available  regarding  client/provider 
satisfaction  with  services  prior  to  the  project,  there  is  no  statistical  evidence 
that  client/provider  satisfaction  increased  because  of  the  project.  However, 
perceptual  data  gathered  from  key  informants  suggested  that  even  though 
they  were  very  frustrated  with  the  services  because  of  technical  difficulties, 
they  could  still  see  the  benefits  of  telehealth  in  the  delivery  of  services. 

■ Even  though  the  program  was  slow  to  get  initiated,  there  is  evidence  that 
some  programs  were  delivered  through  telehealth  that  met  the  special 
needs  of  the  client. 

■ To  date,  very  few  services  have  been  delivered  through  telehealth  in  the 
project  area.  In  spite  of  this,  there  is  evidence  that  telehealth  technology  is 
making  services  more  widely  available  and  accessible  for  the  residents  of 
these  communities. 

■ No  data  were  gathered  to  conclude  whether  or  not  telehealth  has  reduced 
clients’  need  to  travel  outside  of  the  community. 

■ There  is  limited  evidence  to  suggest  that  limited  attempts  have  been  made 
to  use  telehealth  to  facilitate  public/health  stakeholder  participation  in  health 
care  planning. 

■ No  data  were  available  to  conclude  whether  or  not  telehealth  would  reduce 
expenditures  for  individuals/families  and  for  the  KLRHA  staff/employer 
costs.  Key  informants  suggested  that  it  will. 

B Although  many  technical  and  organizational  problems  impeded  this  project, 
most  of  the  key  informants  still  felt  that  telehealth  technology  had  shown 
some  utility  for  delivering  health  services  to  the  remote  communities  in  this 
study. 

Learning 

» Eighteen  months  is  not  long  enough  to  establish  this  type  of  initiative. 

■ Communication  and  collaboration  in  decision  making  between  health  care 
providers  and  the  communities  is  needed. 

■ Abide  by  the  principles  that  have  been  established  for  telehealth  and  learn 
from  other  telehealth  systems  that  have  been  operating  for  many  years. 

■ Adequate  staffing  needs  to  be  in  place  with  adequate  time  for  them  to 
implement  the  program. 

■ Have  a good  plan  in  place. 

■ Telehealth  should  have  had  more  history  and  the  bugs  removed  before 
placing  in  the  northern  communities. 

■ There  is  value  in  using  telehealth  in  the  northern  communities  to  alleviate 
the  difficulties  in  getting  physicians  for  those  regions. 

■ Telehealth  has  value  in  teaching  and  consultation  by  bringing  other 
specialists  to  the  team  of  health  care  providers. 
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■ There  needs  to  be  adequate  training  about  telehealth  and  the  technology 
associated  with  it  for  all  participants. 

Contribution  to  National  Dimensions 

Access 

■ Although  service  interventions  were  few,  it  appears  that  telehealth  increased 
access  to  a limited  range  of  health  services  for  the  northern  communities, 
thereby  reducing  travel  out  of  the  communities 

Quality 

■ Provider  and  client  satisfaction  surveys  that  attest  to  the  quality  of  telehealth 
services  were  not  completed,  however,  from  the  perspective  of  the  key 
informants,  the  project  has  potential  for  delivering  effective,  quality  services  to 
remote  communities. 

Integration 

■ From  the  few  educational  sessions  that  took  place  with  other  stakeholders 
during  the  project,  it  appears,  in  principle,  telehealth  has  potential  to  facilitate 
integrated  health  services. 

Health  Status 

Cost- 

Effectiveness 

■ There  are  no  financial  data  that  identify  the  cost-savings  of  telehealth  to  either 
the  client  or  to  the  KLRHA.  Nevertheless,  in  terms  of  personal  costs,  it  seems 
intuitively  logical  that  each  time  a client  does  not  have  to  leave  his/her 
community  specifically  to  receive  health  services,  the  individual/family 
realizes  a cost  saving.  It  is  much  more  speculative  as  to  whether  the  KLRHA 
realizes  cost-savings  as  a result  of  telehealth. 

Transferability 

■ While  telehealth  technology  has  potential  to  deliver  health  services  to 

underserved  populations  in  remote  communities,  numerous  issues  must  be 
addressed  to  realized  this  potential.  An  RHA  must  be  well  prepared  prior  to 
implementing  telehealth,  and  the  following  should  be  considered:  a 
telehealth  plan  should  be  developed  that  specifies  which  services  will  be 
delivered;  service  providers  must  be  involved  in  the  planning  process  so  that 
they  will  buy-in  to  this  type  of  service  delivery;  the  technology  should  be 
installed  and  tested  prior  to  any  service  delivery;  infrastructure  support 
should  be  in  place,  including  databases  to  track  telehealth  utilization; 
telehealth  staff  should  be  knowledgeable  about  the  use  of  the  technology; 
technical  training  should  be  given  to  service  providers  in  a user-friendly 
manner;  and  mechanisms  should  be  in  place  evaluate  telehealth  outcomes. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

■ Telehealth  holds  great  promise  as  an  alternative  health  care  delivery 
approach  for  providing  services  to  remote  and  isolated  communities  in 
Canada.  However,  the  KLRHA  telehealth  project  demonstrated  that  there  are 
serious  obstacles  that  must  be  overcome  for  this  delivery  approach  to  realize 
its  full  potential.  For  instance,  the  technology  itself  must  be  dependable  and 
user-friendly  or  health  care  managers  and  professionals  will  not  use  it. 
Furthermore,  telehealth  cannot  simply  be  “layered-on”  an  existing  health  care 
system  without  carefully  planning  how  telehealth  service  delivery  will  be 
integrated  with  more  traditional  forms  of  delivery;  moreover,  service  providers 
who  are  expected  to  ultimately  utilize  telehealth  must  be  involved  in  the 
planning  process  and  be  afforded  the  time  and  training  necessary  to 
effectively  incorporate  this  new  approach  into  their  practice. 

Implications  for 
Research  and 
Evaluation 

■ Telehealth  in  the  KLRHA  is  a center-piece  of  the  Alberta  Wellnet  project, 
consequently,  its  on-going  progress  in  experimenting  with  technology  to 
deliver  health/medical  services  should  be  monitored. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Prevention/Promotion:  Community  Development 


Project  Name 

Brooks  Cares  Project 

(Demonstration  Project) 

Project  Description 

The  Brooks  Cares  project  was  designed  to  address  the  growing  health 
needs  (from  a broad  health  determinants  perspective)  associated  with 
rapid  population  expansion  in  the  small  rural  town  of  Brooks.  A 
Participatory  Action  Research/Participatory  Action  and  Learning 
(PAR/PLA)  approach  was  chosen  as  the  strategy  to  help  Brooks’ 
citizens  (established  and  newcomers)  identify  issues,  gather 
information,  develop  solutions,  and  act  on  solutions.  Training  and 
orientation  sessions  were  offered  to  community  members. 

Expected  Outcomes 

1 . Community  members  are  trained  in  PAR/PLA. 

2.  Groups  of  community  members  have  collectively  applied  the  tools 
and  techniques  of  PAR/PLA  to  the  identification  and  analysis  of 
problems,  leading  to  opportunities  for  action  relating  to  primary 
health  care  issues. 

3.  Groups  of  community  members  have  participated  in  processes  of 
decision  making  and  have  developed  plans  for  action. 

4.  Groups  of  community  members  have  implemented  actions  to 
address  opportunities  relating  to  primary  health  care  issues. 

5.  Groups  of  community  members  have  participated  in  monitoring  and 
evaluating  the  outcomes  of  their  actions. 

Methodology  of 

Independent 

Evaluation 

Document  review,  interviews  with  project  team  members,  Steering 
Committee  and  some  community  members  who  had  attended  training 
sessions,  as  well  as  attendance  at  various  Steering  Committee 
meetings. 

Review  of 

Implementation 

Process 

■ Project  objectives  and  expected  outcomes  were  revised  in  June 
1999  (one  year  after  initial  implementation).  A revised  project 
proposal  was  dated  July  1999  and  submitted  to  Alberta  Health  and 
Independent  Evaluation  in  August  1999. 

■ PAR/PLA  consultants  and  a Community  Development  Worker  were 
the  primary  project  implementers.  They  provided  orientation  and 
training  sessions  to  Steering  Committee  members,  some  staff,  and 
community  members. 

■ Because  almost  no  newcomers  were  drawn  into  the  project,  and 
many  workshop  participants  did  not  understand  the  purpose  of  the 
project  and  felt  that  it  was  not  connecting  well  to  either  established 
citizens  or  newcomers;  the  project  was  terminated  at  mid  point. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ A Community  Impact  Study  Team  (CIST)  had  formed  in  the 
community  prior  to  project  implementation.  The  project  Steering 
Committee  was  derived  from  this  group,  some  of  whose  members 
were  strong  advocates  for  the  project  and  the  PAR/PLA  process. 

■ Project  team  members  were  very  strong  advocates  of  the  PAR/PLA 
approach. 

■ Project  goals  and  outcomes  were  narrowed  significantly  in  terms  of 
learning  new  skills,  rather  than  changing  behaviours  of  community 
members. 

Challenges 

■ Articulation  of  PAR/PLA  was  difficult.  Community  members  were 

Brooks  Cares  Project 

(Demonstration  Project) 


fA 
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unfamiliar  with  the  term  and  had  difficulty  understanding  both  the 
concept  and  the  process.  Several  community  members  did  not 
understand  the  nature/purpose  of  the  project.  Community  members 
were  also  skeptical  that  the  process  would  make  any  real  difference 
in  the  community. 

■ Very  small  numbers  of  community  members  were  involved  in  the 
project — almost  none  of  them  were  newcomers. 

■ The  Community  Development  Worker  (project  coordinator)  was 
considered  an  outsider  by  several  community  members,  particularly 
by  other  health  professionals.  The  project  assistant  was  also  a 
relative  newcomer. 

■ The  PAR/PLA  consultant  drew  on  examples  mainly  from  Third 
World  countries.  Some  community  participants  objected  to  this. 

■ The  project  team  expressed  that  because  agencies  were  bound  by 
confidentiality  they  could  not  share  names  of  newcomers  who  were 
in  most  urgent  need  of  assistance. 

■ External  initiation  of  the  project  to  the  community  (counter  to  PAR 
philosophy). 

■ The  project  team  felt  that  PAR  processes  might  take  as  long  as  two 
to  three  years  to  implement  in  communities  like  Brooks.  Since  the 
project  was  initiated  by  someone  other  than  a resident  of  Brooks 
they  felt  that  from  the  outset  the  project  ran  counter  to  the 
philosophy  of  PAR. 

Results 

■ The  project  team  believed  that  training  sessions  provided 
opportunities  to  participants  to  gather  information  from  other 
community  members  (via  conducting  semi-structured  interviews). 

■ Orientation/training  sessions  were  extended  to  about  50  community 
members  (Community  Health  Council  members,  Steering 
Committee,  media,  health  professions,  and  small  numbers  of 
community  members). 

■ Key  issues  emerged  from  working  with  community  members  in  the 
training  sessions  (e.g.,  high  cost  of  living;  lack  of  convenient, 
affordable  public  transportation;  personal  safety;  lack  of  sufficient, 
reliable,  and  affordable  childcare;  lack  of  meaningful  employment 
opportunities;  and  sources  of  appropriate  and  relevant  information). 

■ By  project  end,  a parent’s  group  was  using  PAR  skills  to  address 
lack  of  childcare  in  the  town.  Some  isolated  examples  of  project 
team  members/community  participants  expressed  using  PAR  skills 
in  their  regular  jobs  (e.g.,  Project  Manager,  Lakeside 
representative,  college  instructor). 

■ The  Project  Director  felt  that  trust  was  growing  in  the  process  and 
that  increased  discussion  and  problem  solving  through  information 
gathering  and  analysis  and  planning  had  occurred  in  the  community 
after  project  close. 

Learning 

Much  learning  occurred  about  successful  implementation  of  PAR. 

■ Need  for  clear  definition/explanation  of  the  PAR  process. 

■ Need  for  clearly  defined  project  goals  and  expected  outcomes. 

■ Ability  of  staff  to  make  the  necessary  adjustments  to  the  demands 
of  the  process  and  to  the  special  needs  required  to  make 
meaningful  personal  connections.  In  addition  to  teaching  problem 
solving  skills,  PAR  training  sessions  needed  to  provide 
opportunities  for  community  members  to  socialize,  speak  English, 
and  learn  about  the  Brooks  community.  In  other  words,  settlement 
issues  needed  to  be  addressed. 

■ A clear  workplan  for  PAR  was  required. 
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■ Identification  of  issues  and  solutions  needed  to  come  from 
community  members. 

■ Time  was  needed  to  accomplish  measurable  change. 

■ Time  was  needed  for  Steering  Committee  members  to  become 
familiar  with  the  PAR  process. 

■ The  project  office  needed  to  be  located  near  where  newcomers 
live. 


Contribution  to  National  Dimensions 


Access 

Quality 

Integration 

Health  Status 

Cost- 

Effectiveness 

Transferability 

■ A PAR  strategy  has  merit  as  a successful  strategy  to  empower 
disenfranchised  community  members,  specifically  to  build  skills  in 
identifying  community  issues,  developing  solutions  and  action  plans  to 
address  these  needs  and  issues. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

PAR  is  a strategy  that  has  the  potential  to  build  the  skills  of  community 
members  to  identify  and  address  issues  related  to  health  and  well-being  of 
community  members.  Communities  must  be  socially  and  politically  prepared 
to  accept  the  process,  which  may  take  three  to  five  years  to  implement 
successfully.  Therefore,  funding  support  for  PAR  processes  needs  to  be 
considered  in  light  of  the  readiness  of  communities  and  participants  to 
engage  in  long-term  PAR  processes.  Criteria  for  project  selection  could 
include  the  following  as  key  to  successful  implementation  of  a PAR  process. 

1 . Focus  on  issues  that  are  real  issues  for  community  members,  not  lofty, 
unclear  goals. 

2.  Clarify  project  goals  and  outcomes  to  community  members  before  ! 

starting  the  PAR  process. 

3.  Develop  realistic  project  outcomes.  Examples  might  include 

■ improved  health  status  - mental,  physical,  and  social; 

■ improved  social  support; 

■ increased  integration  of  services;  and 

■ reduced  drug,  alcohol,  and  crime  rates. 

4.  Develop  appropriate  indicators  of  success.  Examples  might  include 

■ increased  knowledge  and  abilities  to  use  support  systems 
appropriately; 

■ increased  trust  and  communication  among  and  between  citizens  and 
human  services  professionals; 

■ increased  opportunities  for  learning,  employment,  recreation  and 
spiritual  development; 

■ widespread  understanding  of  goals  and  processes;  and  1 

■ substantial  voluntary  participation. 

5.  Ensure  that  the  implementation  team  is  clear  from  the  outset  what  the 
project  is  about.  The  team  must  be  able  to  articulate  what  the  purpose 
and  process  is.  Team  members  must  be  a good  match  with  the 
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community. 

6.  Consider  necessary  skills  and  talents  (e.g.,  leadership)  of  an 
experienced  Community  Development  Worker  and  a steering  group  with 
broad  representation  of  organizations’  and  citizens'  needs  that  shares 
the  same  vision  of  development  and  participatory  processes. 

7.  Prepare  the  community  (politically,  socially)  for  the  PAR  process. 
Introduce  the  PAR  to  the  community  in  a way  that  makes  it  real  and 
readily  applicable  to  daily  life. 

8.  Grassroots  support  is  imperative.  Work  with  people  at  their  own  level. 

9.  Support  of  existing  agencies  is  required.  Draw  them  in  as  advocates 
and/or  supporters. 

10.  Expect  challenges/barriers  (e.g.,  skepticism,  jealousy,  vested  interests, 
fear,  communication  limitations — individual  and  systemic). 

1 1 . Funding  support  for  a successful  PAR  process  may  be  required  for  three 
to  five  years  to  establish  a strong  core  group  of  people  who  are  engaged 
and  committed  to  the  process  over  the  long  term. 

Implications  for 
Research  and 
Evaluation 

PAR  has  merit  as  a research/evaluation  methodology  in  community 
development  settings.  However,  skilled  consultants  are  needed,  as  is  time 
for  community  members  to  engage  in  the  process  and  realize  its  benefits. 

Synopsis  of  Independent  Evaluation  Reports 


r Project  Name 

Elnora  Primary  Health  Care  Project 

(Demonstration/Evaluation  Project) 

Project 

Description 

The  Elnora  demonstration  and  evaluation  project  endeavored  to  develop, 
implement,  and  evaluate  a model  that  combined  primary  health  care  and 
community  development  principles  in  the  creation  of  a rural  Health  Centre 
and  the  addition  of  a Nurse  Practitioner  at  the  Centre.  The  model  was  used 
to  provide  Elnora  area  residents  with  affordable,  accessible,  effective,  and 
acceptable  primary  health  care  services.  The  project  was  developed  as 
part  of  a broader  Healthy  Communities  Initiative  (HCI),  which  involved  five 
sites  in  the  David  Thompson  Health  Region. 

Expected 

Outcomes 

1 . Implementation  and  evaluation  of  a PHC  model  designed  specifically 
for  rural  communities. 

2.  Local  ownership  over  the  processes,  decisions,  Elnora  Health  Centre, 
and  community  by  Elnora  residents. 

3.  Renewed  sense  of  empowerment  and  hope  over  individual  and 
community  health  by  Elnora  residents. 

4.  Perception  by  Elnora  area  residents  that  their  identified  priority  health 
service  needs  (i.e.,  for  access  to  a local  physician,  enhanced  first 
response  and  increased  services  to  seniors)  are  being  addressed. 

5.  Awareness  by  Elnora  area  residents  of  the  health  and  human  services 
available  in  the  Centre,  their  community,  and  the  DTHR  and  how  to 
access  these  services. 

6.  Use  by  Elnora  area  residents  of  the  primary  health  care  (PHC)  services 
developed  at  the  Centre. 

7.  Evidence  the  HCI  and  PHC  teams  have  worked  together  to  ensure  that 
effective  primary  health  care  services  have  been  planned, 
implemented,  and  evaluated. 

Methodology  of 

Independent 

Evaluation 

■ This  was  an  “evaluation  of  an  evaluation”  and  the  independent 
evaluators  relied  on  locally  developed  data.  These  data  were 
supplemented  by  periodic  contact  with  the  local  evaluators  and  project 
team  and  there  were  limited  efforts  to  generate  new  data  for  purposes 
of  validation  of  local  reports. 

■ In  the  end,  the  independent  evaluator  was  limited  to  document  review, 
review  of  and  advice  on  the  local  team’s  methodology,  and  three  site 
visits  including  two  focus  groups — one  with  former  members  of  the  HCI 
sub-committee,  the  other  with  primary  health  project  team  members. 
Additionally,  there  were  topical  discussions  with  the  internal  evaluation 
team  (e.g.,  detailed  discussion  of  the  interim  evaluation  report  and 
cooperative  development  of  activities  such  as  focus  groups  and  the 
interface  of  future  plans),  and  a few  other  members  of  DTHR 
administration,  as  well  as  contact  with  the  EMT. 

■ The  project  involved  two  levels  of  evaluation  with  different,  but 
overlapping  focuses  and,  in  order  to  better  coordinate  the  efforts  of  the 
two  evaluation  teams,  a “roles  and  responsibilities  agreement”  was 
developed  by  DTHR  in  consultation  with  AHW  and  the  independent 
evaluators.  For  the  most  part,  the  document  served  as  a useful  tool  for 
avoiding  duplication,  however,  at  times,  a rigid  interpretation  of  the 
document  impeded  the  efforts  of  the  independent  evaluators  to  conduct 
their  work. 

Review  of 

Implementation 

Process 

Initial  Process 

The  Elnora  project  developed  out  of  an  ongoing  process  of  community 
development  facilitated  by  the  region’s  Healthy  Communities  Initiative 
(HCI),  funded  under  the  Action  for  Health  envelope.  In  1995,  the  Elnora 
hospital  was  closed.  In  early  1998,  Elnora  residents  established  an  Action 

Elnora  Primary  Health  Care  Project 

(Demonstration/Evaluation  Project) 
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for  Health  Committee  and  this  group  submitted  a successful  proposal 
(1998)  to  the  DTHR  to  join  the  HCI.  Upon  approval  of  the  HTF  project,  the 
DTHR  established  a Primary  Health  Care  Project  Subcommittee  under  the 
HCI  Steering  Committee. 

Guidina  Principles 

The  Elnora  project  was  guided  by  six  principles  of  primary  health  care  and 
community  development,  and  project  goals  and  objectives  were  related  to 
these. 

Model  Description  and  Components 

The  Elnora  PHC  project  model  had  three  components:  the  PHC  team,  PHC 
services,  and  community  action.  This  model  was  also  influenced  by 
complex  external  factors  such  as  the  DTHR  and  other  health  and  human 
services  and  other  community  initiatives  in  the  Elnora  area. 

Facilitators/ 
Challenges  to 
Implementation 

The  independent  interim  evaluation  report  identified  six  issues  which  arose 

from  the  review  of  the  June  1999  internal  evaluation  report  and  each  of 

these  were  addressed  in  the  final  evaluation  report: 

■ Issue  1 - the  question  of  the  status  of  the  seniors’  services  feasibility 
study.  The  study  was  finally  presented  to  the  community  in  December 
1999. 

■ Issue  2 - continuity  in  the  evaluations,  given  staff  turnover.  In  the 
summer  1999,  a new  research  and  evaluation  associate  was 
appointed. 

- Issue  3 -internal  and  independent  evaluation  teams  agreed  on  a 
strategy  for  improving  the  sampling  procedures  for  the  second  survey 
in  February  2000. 

■ Issue  4 - the  understanding  and  congruency  of  the  concepts  and  terms 
being  employed  in  the  project.  This  topic  was  raised  by  DTHR 
administration  with  the  DTHR  Board  in  fall  1999.  This  was  due  to  a lack 
of  shared  understanding  of  terms  (e.g.,  accessible,  affordable)  by 
community  members  and  the  project  team  as  well  as  a lack  of 
understanding  by  community  members  of  the  model  used  in  the  project 

■ Issue  5 - follow-up  on  the  internal  evaluation  report  suggestion  that 
more  attention  be  paid  to  evaluation  of  group  dynamics  between  the 
HCI  and  PHC  project.  The  independent  evaluators  forwarded  literature 
to  the  internal  evaluators  on  the  topic  of  team  dynamics. 

■ Issue  6 - question  of  cost-effectiveness.  Independent  evaluators 
recommended  a contact  to  assist  DTHR  with  the  issue  of  cost- 
effectiveness.  However,  in  the  end,  all  aspects  of  costing  proved 
problematic  for  the  project. 

Results 

Primarv  health  care  team 

■ The  project  appeared  to  have  involved  teams  within  teams,  or  three 
overlapping  teams.  The  majority  of  the  “team”  was  already  co-located 
at  the  Elnora  Health  Centre  prior  to  the  development  of  the  project. 
Consequently,  the  innovative  aspect  of  this  project  was  the  addition  of 
the  Nurse  Practitioner  (NP),  physicians,  and  pharmacist.  Users  defined 
the  NP  role  (e.g.,  minor  illnesses,  exams,  emergencies,  and  health 
education). 

■ Staff  perceptions  suggested  that  there  were  three  levels  of 
involvement:  the  “core”  team,  “peripheral”  members,  and  members  who 
consulted  with  other  team  members. 

■ Team  members  viewed  the  NP  as  pivotal  to  the  project.  Collaboration 
of  physicians  was  also  viewed  as  important  to  encouraging  team 
cohesion. 
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■ Although  team  member  interviews  revealed  a relatively  strong  sense  of 
belonging,  other  aspects  of  the  internal  evaluation  indicated  a 
decreasing  sense  of  team  support,  shared  leadership  and  consensus 
decision  making  during  the  course  of  the  project. 

■ The  physicians  were  not  well  integrated  with  the  rest  of  the  team.  If  this 
project  continues  with  HIF  funding,  the  NP  will  be  located  with  the 
physicians  in  Trochu  and  this  will  have  implications  for  continuation  of 
service  in  Elnora. 

Primarv  health  care  services 

■ Many  of  the  services  that  were  provided  at  the  Elnora  Health  Centre 
during  the  project  were  already  in  place  at  the  onset  of  the  project. 

Three  additional  services  were  added  as  a result  of  the  project: 
ambulatory  care,  emergency  response  management,  and  injury 
prevention,  and  the  NP  played  a major  role  in  the  provision  of  these 
services. 

■ The  major  innovation  in  service  delivery  occurred  through  the  role  of 
the  NP. 

■ 70%  of  services  were  accessed  by  women 

Communitv  action 

■ A variety  of  structures  were  established  that  facilitated  the  development 
of  the  project:  HCI  Steering  Committee,  Action  for  Health  Committee; 
PHC  Subcommittee;  and  a variety  of  ad  hoc  committees.  The 
independent  evaluators  were  uncertain  of  the  extent  to  which  the 
community  was  involved  in  project  implementation. 

■ The  community  facilitator  felt  uncomfortable  in  her  role  of  conveying  the 
views  of  the  community  to  project  staff. 

■ The  PHC  Subcommittee  members  indicated  a sense  of  ownership  and 
experienced  a lot  of  satisfaction  with  the  project. 

■ The  PHC  Subcommittee  had  minimal  contact  with  the  project  after  the 

arrival  of  the  NP.  ! 

■ Nearly  half  of  project  team  members  regarded  the  link  of  the  project  to 
the  HCI  as  weak.  Slightly  fewer  saw  differing  agendas  of  the 
community  and  the  team  as  a barrier  to  the  project. 

Learning 

■ While  the  HCI  process  is  laudable  for  providing  a forum  for  the 
identification  of  community  needs  and  the  development  of  options  to 
address  those  needs,  the  extent  to  which  a partnership  between  the 
community  and  the  project  team  was  sustained  was  not  clearly 
evidenced  through  the  internal  evaluation  or  through  data  gathered 
through  the  independent  evaluators’  focus  groups.  A variety  of 
explanations  for  this  weakness  in  linkage  are  described  throughout  the 
internal  evaluation  report. 

■ Integrating  community  development  principles  and  enhancing  primary 
health  care  confused  lay  and  professionals  alike.  Constant  reflection  on 
roles  was  needed,  including  formal,  continuous,  and  effective 
communication. 

Contribution  to  National  Dimensions 

Access  ■ Survey  data  from  the  internal  evaluation  indicated  that  overall  . 

awareness,  and  subsequent  utilization  of  services  offered  at  the  Elnora 
Health  Centre  increased  during  the  course  of  the  project.  i! 
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B Quality 

■ The  framework  for  quality  used  in  project  was  based  on  the  assumption 
that  community  participation  and  group  processes  which  make  beneficial 
changes  for  individuals  and  the  community  are  in  themselves  health 
enhancing.  This  Framework  is  based  on  the  Canadian  Council  on  Health 
Services  Accreditation  model  that  focuses  more  on  professional  and 
managerial  behaviour.  However  they  both  emphasize  meeting 
client/community  expectations  and  working  in  partnership. 

■ The  only  measure  of  quality  related  to  client  satisfaction.  Overall, 
satisfaction  with  services  increased  during  the  course  of  the  project,  and 
much  of  this  satisfaction  revolved  around  the  scope  of  practice  and 
personal  attributes  of  the  Nurse  Practitioner. 

■ Increased  satisfaction  was  also  reported  for  emergency  response 
services.  Satisfaction  increases  for  these  services  may  be  attributed  to 
the  fact  that  the  community  had  previously  lost  hospital  and  physician 
services,  and  these  “new”  services  filled  part  of  the  gap. 

Integration 

■ A broad  range  of  services  were  co-located  and  coordinated  through  the 
Elnora  Health  Centre.  However,  the  extent  to  which  services  were 
integrated  as  opposed  to  coordinated  or  simply  co-located  is  difficult  to 
determine.  The  internal  evaluation  reported  that  the  lack  of  team-building 
infrastructure  and  insufficient  time  devoted  to  team  building  may  have 
limited  the  extent  of  integration  during  the  project. 

■ During  the  project  the  NP  departed  and  pending  plans  to  relocate  the  NP 
to  the  physician  practice  in  Trochu  may  affect  the  capacity  to  further  build 
an  integrated  team  at  the  Elnora  site. 

Health  Status 

■ Generally,  changes  to  long-term  health  status  of  individuals  and 

communities  cannot  be  addressed  within  the  time  frame  of  the  project. 
However,  the  independent  evaluators  recognize  the  potential  impact  the 
HCI  process  and  the  primary  health  care  project  may  have  had  on  the 
collective  psyche  of  the  community  in  terms  of  engagement  in  initial 
decision  making  about  the  project  and  the  resultant  sense  of 
achievement  and  ownership.  The  health  of  the  community  might  be 
further  impacted  by  the  decreased  level  of  anxiety  about  the  availability 
of  basic  health  services  in  the  Elnora  community. 

Cost- 

Effectiveness 

Transferability 

■ The  way  in  which  the  Elnora  project  unfolded  is  problematic  from  the 
perspective  of  transferability  and  generalizability.  In  the  case  of  the 
Elnora  project,  the  HCI  process  might  be  transferable  to  other 
jurisdictions,  but  results  would  likely  vary.  The  Elnora  community  was 
motivated  by  the  closure  of  its  local  hospital.  In  this  case  the  community 
was  very  accepting  of  the  Nurse  Practitioner  solution  to  losing  a hospital. 

■ The  service  delivery  model  for  Elnora  might  be  transferable  to  other  rural 
communities,  but  circumstances  might  vary  from  one  context  to  the  next. 
In  this  context  providers  were  already  working  together.  The 
interpersonal  skills  of  the  Nurse  Practitioner  were  also  key  to  facilitating 
greater  team  integration  and  community  acceptance  of  the  services. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

■ Regulatory  and  funding  mechanisms  within  Alberta  may  limit 
transferability  of  the  Nurse  Practitioner  concept  (i.e.,  funding  NPs 
through  regional  budgets  while  continuing  to  fund  physicians  through 
provincial  budgets  does  not  make  NPs  attractive  to  regions  as  cheaper 
alternatives  to  physicians,  at  present).  Current  professional  regulations 
need  to  be  changed  to  allow  NPs  to  practice  more  readily  in  a variety  of 
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settings  with  a variety  of  providers. 

■ A significant  issue  relates  to  the  sustainability  of  the  project.  The  project 
developed  as  a result  of  HTF  funding.  All  the  major  stakeholders, 
especially  the  community,  were  concerned  about  how  all  of  their  efforts 
to  develop  and  implement  the  project  would  be  sustained  beyond  the 
HTF  mandate.  While  DTHR  has  acquired  new  funding  through  the 
Health  Innovation  Fund,  the  new  project  appears  to  be  fundamentally 
different  from  the  original  HTF  project.  Specifically,  the  Elnora 
community  believes  it  is  losing  the  very  service  the  community  fought  so 
hard  to  get.  Transferability  will  require  developing  provincial  and  local 
structures  and  processes  to  create  a much  more  stable  environment 
than  was  present  in  this  project. 

Implications  for 
Research  and 
Evaluation 

It  is  important  to  verify  internal  evaluation  findings  by  independent  review. 
Survey  samples  must  be  representative  of  the  number  of  community 
residents  before  statements  can  be  generalized  to  the  entire  community. 
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| Project  Name 

Evaluation  of  Healthy  Okotoks  Project 

(Evaluation  Project) 

Project  Description 

Headwaters  Regional  Health  Authority  initiated  the  Healthy 
Communities  Project  in  September  1996  as  part  of  its  commitment  to 
deliver  a wellness-focused  health  care  system.  A Healthy  Okotoks 
Coalition  (HOC)  formed  as  a local  development  process  to  encourage 
citizen  involvement  and  enhance  capacity  of  indigenous  leadership  to 
address  local  concerns.  Over  fifty  partners  representing  multiple 
sectors,  agencies,  and  age  groups  comprised  the  Coalition. 

A grassroots  community  approach  was  adopted  to  problem  solve  and 
build  linkages.  The  Coalition  focused  its  efforts  on  health  promotion 
and  disease  prevention  through  intersectoral  collaboration  and 
programming. 

The  Healthy  Okotoks  project  consisted  of  two  phases.  Phase  1 was 
devoted  to  exploring,  describing,  and  evaluating  the  process  of 
creating  a healthy  community  particularly  with  respect  to  assessing  the 
extent  to  which  implementation  respected  fundamental  values  and 
principles  of  health  promotion,  empowerment,  and  community 
development.  Phase  2 concentrated  on  developing  a template  for 
tracking  Coalition  processes  and  outcomes  (using  different  types  of 
logs),  and  surveying  Coalition  members'  satisfaction,  significance  of 
changes  made  by  the  Coalition,  and  Coalition  strength. 

Expected  Outcomes 

1 . Development  of  a set  of  outcomes  and  process  measurement 
tools. 

2.  Identification  of  those  processes  and  events  that  act  as  barriers 
and  facilitate  certain  outcomes,  including  those  that  may  be 
unintended. 

3.  Application  of  the  findings  to  improve  the  processes  and  outcomes 
of  the  Healthy  Okotoks  Project  and  throughout  the  Headwaters 
Health  Region  and  make  Healthy  Okotoks  Coalition  a better 
organization. 

4.  Development  of  a framework  for  the  evaluation  of  future  Healthy 
Communities  projects. 

Methodology  of 

Independent 

Evaluation 

The  Independent  Evaluation  of  the  Healthy  Okotoks  project  involved  a 
peer  review/supervision  process  of  internal  evaluation  activity  which 
involved  the  following: 

Phase  1 

Qualitative  research  metjiods  were  used  to  conduct  Phase  1.  Data 
were  gathered  from  18  members  of  the  Healthy  Okotoks  Coalition  (in- 
depth  interviews),  11  Coalition  meetings  (participant  observation),  and 
independent  assessment  of  documents  and  meeting  notes. 

Phase  II 

Quantitative  and  qualitative  methods  were  used  in  evaluating  Phase  II 
of  the  Healthy  Okotoks  Project.  A system  of  logs  were  used  to 
document  system  or  community  changes,  gather  information  about 
educational  or  service  programs  provided  by  the  Coalition,  track 
amount  of  media  attention  received  by  the  Coalition,  and  track 
resources  secured.  In  addition,  evaluators  reviewed  Healthy  Okotoks 
Coalition  meeting  minutes.  Minutes  were  coded  according  to 
objectives,  activities,  dates,  outcomes,  and  process  measures.  A 
survey  was  developed  to  measure  Coalition  members’  satisfaction. 

Evaluation  of  Healthy  Okotoks  Project 

(Evaluation  Project) 
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Review  of 

Implementation 

Process 


There  were  two  distinct  phases  to  the  evaluation  of  the  HOC.  The  first 
phase  was  devoted  to  exploring,  describing,  and  evaluating  the 
process  of  creating  a healthy  community,  while  the  second  phase  j 
concentrated  on  developing  a template  for  tracking  coalition  processes 
and  outcomes,  and  surveying  Coalition  members'  satisfaction 

Phase  1 

As  an  evaluation  project,  Phase  I of  the  implementation  process  was  a 
process  evaluation  which  addressed  the  following  questions: 

1 . What  is  the  process  of  creating  a healthy  community  in  a rural 
setting? 

2.  What  is  the  role  of  both  local  government  and  the  Regional  Health 
Authority  in  the  process  of  creating  a healthy  community  in  this 
context? 

3.  Has  the  process  of  creating  a healthy  community  in  this  context 
been  implemented  as  intended,  particularly  with  respect  to  the 
fundamental  values  and  principles  of  health  promotion, 
empowerment,  and  community  development? 

Data  were  collected  through  two  primary  means:  semi-structured 
interviews  with  individual  Coalition  members,  participant  observation  of 
Coalition  meetings — both  of  which  were  conducted  by  the  internal 
evaluator. 

Phase  2 

The  purpose  of  Phase  2 was  twofold — monitoring  and  evaluating  the 
Coalition  to  enhance  understanding  of  the  HOC  and  its  impact  and  to 
improve  the  functioning  of  the  HOC.  Phase  2 aimed  to 

■ apply  a set  of  outcome  and  process  measurement  tools  for 
community  coalitions; 

■ verify  the  applicability  of  a methodology  for  monitoring  and 
evaluating  community  coalitions  to  the  HOC; 

■ identify  processes  and  events  that  act  as  barriers  and  facilitate 
certain  outcomes,  including  those  that  may  be  unintentional; 

■ develop  a framework  or  template  for  the  evaluation  of  future 
healthy  communities  projects  to  be  implemented  within  the  region, 
the  province  and  Canada;  and 

■ measure  membership  satisfaction  with  the  help  of  a survey. 

A mixed  research  design  using  both  qualitative  and  quantitative 
research  methods  was  used  for  Phase  2. 

The  evaluators  modified  an  existing  and  well-tested  log  (Fawcett  et  al., 
1995)  to  track  the  processes  undertaken  by  the  Coalition.  This  log  was 
based  on  process  and  outcome  measurement  tools  for  community 
coalitions  (i.e.,  event  logs,  ongoing  services  provided  logs,  media 
coverage  logs,  and  resources  generated  logs).  Event  logs  were  used 
to  record  activities  that  have  the  goal  of  making  systems  changes  or 
community  changes.  The  ongoing  services  provided  log  gathered 
information  about  educational  or  service  programs  provided  by  the 
group.  The  media  coverage  log  tracked  the  amount  of  media  attention 
the  coalition  received.  The  resources  generated  log  tracked  in-kind 
and  monetary  resources  secured  by  the  initiative. 

Two  internal  evaluators  reviewed  HOC  meeting  minutes  dating  from 
February  1997  to  June  1999.  Each  evaluator  reviewed  the  minutes 
and  coded  the  minutes  according  to  objectives,  activities,  dates, 
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outcomes,  and  process  measure. 

A constituent  survey  was  used  to  collect  quantitative  data  on  Coalition 
member’s  satisfaction,  the  significance  of  the  changes  that  the 
initiative  has  made,  and  Coalition  strength. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

There  is  little  question  that  the  Healthy  Okotoks  project  was  able  to 
meet  all  expected  outcomes  identified  at  the  beginning  of  the  project. 
The  research  interests  of  the  evaluators  allowed  the  project  to  conduct 
a very  comprehensive  assessment  of  Coalition  process,  activities,  and 
outcomes  for  a reasonable  cost.  A member  of  the  Okotoks  Town 
Council  was  a Coalition  member.  This  linkage  created  a direct 
connection  to  municipal  government  and  provided  formal  and  informal 
support  to  the  project. 

Challenges 

■ Differences  in  perceived  purposes  of  the  Coalition  increased  the 
difficulty  in  project  startup  and  attracting  a new  and  broad 
membership  base. 

■ Difficulty  in  identifying  the  needs  of  the  under-served  and 
disenfranchised  individuals  and  groups  in  the  community. 

■ Minutes  of  meetings  were  not  comprehensive  making  their  review 
difficult.  Another  challenge  was  identifying  objectives,  since  they 
were  not  always  clearly  stated  in  the  meeting  minutes. 

Results 

The  main  outcome  of  the  implementation  was  a conceptual  framework 
describing  the  Coalition  building  process.  Another  key  outcome  is  the 
development  of  a framework  for  the  Coalition  to  track  its  progress, 
which  can  also  be  used  for  evaluation  purposes,  as  it  was  used  to 
evaluate  the  HOC. 

Community  changes  influenced  in  part  or  whole  by  the  HOC 
constituted  key  project  outcomes: 

■ the  creation  of  a Community  Garden, 

■ fall  clean  up  - annual  event; 

■ FAS  awareness  and  Healthy  Babies  pilot  project; 

■ increased  awareness  of  FAS  among  8th  graders; 

■ lead  free  paint  for  playground  equipment; 

■ youth  curfew  implemented  for  youth  under  15  year  olds; 

■ formation  of  a Youth  Action  Group; 

■ alteration  of  the  roller-blade  bylaw; 

■ skateboard  park  approved  by  Town  Council; 

■ youth  positions  added  to  various  Okotoks  Town  Council 
Committees;  and 

■ support  for  Environmental  Week. 

An  unexpected  outcome  of  the  HOC  was  that  it  acted  as  a social 
support  network  to  some  Coalition  members. 

Phase  1 Findinas 
Coalition  Framework 

A Coalition  purpose  was  defined  from  two  different  perspectives.  The 
first  perspective  suggested  that  the  Coalition’s  purpose  was  providing 
the  community  with  expertise  that  could  “assist,  facilitate,  and  guide 
others  in  the  community  in  attempting  to  influence  their  environment  to 
improve  health.”  The  second  view  suggested  that  the  Coalition  was 
“simply  a mechanism  for  people  and  organizations  to  work  together  on 
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a variety  of  specific  issues  for  the  health  of  the  community.”  While 
these  perspectives  were  not  necessarily  incompatible,  they  did  create 
some  ambiguity  around  the  Coalition’s  purpose,  which  in  turn,  gave 
some  members  difficulty  in  discerning  which  direction  to  take  and  what 
projects  to  launch. 

Coalition  Function 

The  Coalition  depended  upon  informal  structures  for  needs 
identification.  Almost  all  concerns  were  brought  forward  for 
consideration  by  the  membership.  They  experienced  some  difficulty  in 
identifying  the  needs  of  the  under-served.  The  formal  decision  making 
process  was  consensus.  However,  not  all  decisions  were  firm  and 
members  often  reversed  or  reviewed  decisions  based  upon  member 
skepticism  or  the  need  to  acquire  additional  information.  Issues  of 
sustainability  emerged.  They  centred  on  the  relatively  small  core  group 
within  the  Coalition,  time  commitment  required  by  members,  lack  of 
knowledge  about  the  Coalition  by  the  broader  community,  and  limited 
financial  resources.  In  summary,  the  Healthy  Okotoks  Coalition  was 
driven  by  a shared  vision  of  a healthy  community  and  developed 
workable  functions  to  address  and  achieve  the  vision.  The  character  of 
the  Coalition,  however,  reflected  the  character  of  its  members. 

Activities  were  driven  by  personal  interest  or  what  the  group  believed 
to  be  of  public  interest. 

Phase  2 Findings 

The  Healthy  Okotoks  Coalition  was  successful  in  achieving  24 
community  changes.  In  the  experience  of  the  Coalition,  a great  deal  of 
time  and  effort  had  gone  into  planning  products  and  community 
actions  before  a community  change  took  place.  For  example,  it  took 
one-and-one-half  years,  31  planning  products,  and  15  community 
actions  to  achieve  one  community  change. 

Group  composition  appeared  to  be  reasonably  strong.  The  Coalition 
survey  suggested  that  members  felt  that  the  Coalition 

■ was  weak  in  number  and  variety  of  members  in  the  Coalition; 

■ was  weak  in  its  ability  to  access  a variety  of  resources; 

■ did  not  have  sufficient  members  to  conduct  Coalition  activities; 

■ was  moderately  secure  in  that  members  believed  benefits 
outweigh  associated  costs  of  membership; 

■ was  moderately  secure  in  that  members  worked  together  yet  had 
sufficient  autonomy; 

■ was  strong  in  leadership;  and 

■ was  strong  in  experience  and  skill. 

While  the  structure  of  the  Coalition  appeared  to  be  reasonably  strong, 
membership  suggested  that  they  could  benefit  from  improved 
processes  for  resolving  internal  and  agency  conflict. 

The  core  membership  in  the  Coalition  was  very  satisfied  with  the 
progress  and  accomplishments  of  the  Coalition.  They  were  happy  with 
the  Coalition’s  ability  to  attract  influential  members  of  the  community. 
They  were  not  satisfied  with  the  Coalition’s  ability  to  reach  out  to 
ethnic  groups  and  visible  minorities. 
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1 Learning 

The  willingness  of  the  Coalition  to  share  its  experiences  provided  a 
foundation  for  understanding  the  process  of  creating  a healthy 
community  in  a rural  context.  In  the  opinion  of  the  evaluators,  this 
understanding  is  critical  to  further  development  of  healthy  communities 
projects  in  this  and  other  contexts. 

Contribution  to  National  Dimensions 

Access 

The  number  of  projects  initiated  and  supported  by  the  Coalition  suggests 
that  members  of  the  community  now  have  better  access  to  health  related 
activities — that  is,  the  Coalition  has  facilitated  access  to  activities  that 
promote  healthy  living  including 

■ the  creation  of  a Community  Garden; 

■ fall  clean  up  - annual  event; 

■ FAS  awareness  and  Healthy  Babies  pilot  project; 

■ increased  awareness  of  FAS  among  8th  graders; 

■ lead  free  paint  for  playground  equipment; 

■ youth  curfew  implemented  for  youth  under  15  year  olds; 

■ formation  of  a Youth  Action  Group; 

■ alteration  of  the  roller-blade  bylaw; 

■ skateboard  park  approved  by  Town  Council; 

■ youth  positions  added  to  various  Okotoks  Town  Council  committees; 
and 

■ support  for  Environmental  Week. 

Quality 

Integration 

■ The  Healthy  Okotoks  Coalition  demonstrated  intersectoral  collaboration 
within  the  town  of  Okotoks,  between  key  stakeholder  groups, 
organizations,  and  the  Headwaters  Health  Authority.  For  example,  the 
Healthy  Okotoks  Coalition  collaborated  with  an  environmental  lobby  to 
encourage  responsible  pesticide/herbicide  use  in  the  community,  and  it 
linked  with  local  expertise  to  increase  the  awareness  of  FAS. 

Health  Status 

■ The  Healthy  Okotoks  Coalition  includes,  as  part  of  its  definition  of 

primary  health  care,  ideals  of  a process  of  health  development  aimed  at 
providing  people  with  a level  of  health  that  permits  them  to  lead  a 
socially  and  economically  productive  life,  to  realize  aspirations,  to 
satisfy  needs,  and  change  and  cope  with  the  environment.  The  HOC 
recognized  and  addressed  the  determinants  of  health  including  social, 
cultural,  and  economic  factors  that  affect  the  health  status  of  individuals 
and  the  community  of  Okotoks.  The  Healthy  Okotoks  Coalition  has 
provided  a means  to  the  community  to  contribute  and  help  toward  the 
creation  of  a community  garden,  and  it  has  helped  to  create  safer 
environments  for  children.  The  Healthy  Okotoks  Coalition  has  provided 
a means  to  empower  youth  to  address  issues  of  particular  interest  to 
them  (youth  curfew,  skateboard  park,  roller-blading  bylaw,  youth 
position  on  certain  Town  Council  committees).  Among  other 
accomplishments,  the  Coalition  has  increased  awareness  of  FAS,  and  it 
has  helped  create  an  annual  fall  clean-up  day. 

Cost- 

Effectiveness 

■ The  Healthy  Okotoks  Project  was  included  in  a sustainability  study 
conducted  within  the  umbrella  Alberta  Primary  Health  Care  Project. 
Detailed  costing  of  the  project  is  outlined  in  that  study.  Therefore,  this 
study  has  provided  effectiveness  and  cost  data  that  can  be  used  for 
comparison  purposes.  These  data  will  assist  an  economic  evaluator  in 
determining  the  “worth”  of  this  approach  relative  to  the  approach  that 
was  in  place  before  the  study  began  or  to  an  alternate  approach. 
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1 Transferability 

■ The  Healthy  Okotoks  Coalition  evaluation  results  can  be  transferred  to 
and  used  as  a guide  to  creating  other  healthy  community  coalitions. 

These  results  provide  an  understanding  of  what  it  takes  to  develop  a 
healthy  communities  coalition. 

■ The  evaluation  log  can  be  used  as  a template  the  Healthy  Okotoks 

Coalition  and  other  coalitions  to  track  process  and  progress.  i 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

At  this  point  local  government  (Municipal  and  Regional  Health  Authority) 
support  the  Coalition  and  have  not  imposed  an  outside  will  on  it.  The 
Coalition  has  made  progress  in  following  the  fundamental  values  and 
principles  of  health  promotion,  empowerment,  and  community  development. 
What  has  not  been  realized  is  the  extension  of  the  Coalition  into  the 
broader  community  to  the  degree  the  membership  feels  is  desirable  and 
necessary  to  be  representative  of  community  health  need,  particularly  of 
marginalized  groups. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Prevention/Promotion:  Early  Intervention 


Project  Name 

Healthy  Families  - Capital  Health  Region 

(Demonstration  Project) 

Project  Description 

The  Healthy  Families  program  is  a voluntary  home  visitation  program 
which  provides  long-term,  intensive  services  to  families  who  have 
economic  and  social  factors  which  place  them  at  higher  risk  for  poor 
outcomes  for  their  children.  The  program  was  based  on  the  Healthy 
Families  America  program  introduced  in  the  United  States.  The 
program  is  delivered  through  a partnership  of  Capital  Health  with  three 
community-based  agencies  in  Edmonton  (Terra  Association,  Bent 
Arrow  Traditional  Healing  Society,  and  Norwood  Child  and  Family 
Resource  Centre).  A Steering  Committee  with  broad  membership 
including  some  from  the  Success  by  Six®  Council  of  Partners.  Initial 
screening  was  conducted  by  a variety  of  agencies  that  deal  with 
pregnant  or  new  mothers.  Self-referrals  were  also  accepted.  If  families 
agreed  to  participate,  they  were  enrolled  and  linked  with  a Family 
Visitor  who  made  weekly  visits  to  provide  education  and  skill 
development  in  parenting,  and  to  provide  links  to  community  resources. 
A community  health  nurse  was  an  integral  member  of  the  Healthy 
Families  team  at  each  community  agency  to  link  families  to  public 
health  services,  and  other  service  providers  in  the  health  system. 

Expected  Outcomes 

1 . Parents  have  increased  knowledge  and  use  of  community 
supports. 

1.1  To  increase  the  number  of  positive  supports  in  the  client’s  life. 

2.  Parents  improve  in  their  personal  development. 

2.1  To  see  an  increase  in  the  parent’s  ability  to  set  and  meet 
personal  goals. 

2.2  To  increase  parent’s  knowledge  of  child  development. 

2.3  To  increase  the  length  between  subsequent  pregnancies. 

3.  Children  have  healthy  growth  and  development. 

3.1  Healthy  growth  and  development  of  the  child  prenatally, 
physically,  socially,  emotionally,  and  cognitively. 

3.2  Mother’s  maintenance  of  a healthy  pregnancy. 

4.  Parents  develop  positive  parent/child  interactions. 

4.1  To  facilitate  the  development  of  positive  parenting  skills. 

4.2  To  increase  the  parent’s  knowledge  and  skills  related  to  child 
rearing  and  behaviour  management. 

Methodology  of 

Independent 

Evaluation 

Project  Implementation 

Participatory,  collaborative,  coordinated  approach 

■ The  Healthy  Families  Program  in  the  Capital  Region  was  primarily 
funded  by  Health  Canada  and  Alberta  Health  and  Wellness 
(Primary  Health  Care  Project)  and  Justice  Canada  (National  Crime 
Prevention  Investment  Fund).  A Healthy  Families  Evaluation 
Committee  was  formed  providing  coordination  between  the 
activities  of  the  two  evaluations. 

■ The  Healthy  Families  Evaluation  Committee  developed  a set  of 
principles  for  the  evaluation  based  on  a participatory  approach.  The 
two  evaluation  teams  worked  collaboratively,  sharing  information 
and  instruments  where  appropriate. 

■ The  purpose  of  the  HTF  evaluation  was  to  gather  information  on 
process  and  outputs  (Dec/98-June/00). 

■ The  purpose  of  the  Justice  Canada  evaluation  was  to  gather 
information  on  outcomes  (Jan/99-Dec/01). 
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Independent  Evaluation  ! 

Independent  evaluation  was  comprised  of  several  components: 

■ Phase  1:  Outcomes  Readiness  Assessment, 

■ Phase  II:  Descriptive  Process  Analysis, 

■ Phase  III:  Process  Evaluation, 

■ Phase  IV:  Analysis  and  Synthesis  of  Outcomes  Measurement 
Process,  and 

■ Phase  V:  Project  Final  Report  Preparation. 

Key  data  collection  methods  included  ! 

■ standardized  measures  and  scales  (nine  instruments): 

■ review  of  data  from  Alberta  Health  and  Wellness,  Capital  Health, 
and  Child  Welfare; 

■ site  visits; 

■ interviews  with  senior  regional  staff  and  other  key  informants; 

■ focus  groups  with  community  partners; 

■ focus  groups  with  parents;  and 

■ client  satisfaction  survey. 

Review  of 

Implementation 

Process 

■ Capital  Health  began  investigating  the  potential  of  the  Healthy 
Families  America  program  in  1997.  A literature  review  was 
undertaken  and  CHA  representatives  attended  a Healthy  Families 
America  conference. 

■ Healthy  Families  committee  formed  (March  1998).  The  Success  by 
Six®  Council  of  Partners  became  the  umbrella  organization  for  the 
promotion  and  development  of  the  program  in  the  Capital  Health 
Region.  The  Success  by  Six®  Council  of  Partners  is  a collaboration 
of  more  than  20  community  partners  with  the  aim  of  supporting 
healthy  development  for  children.  Other  committees  were  formed  to 
deal  with  implementation  and  budget  issues. 

■ The  funding  proposal  was  submitted  to  HTF  (May  1998)  and 
approval  secured  (July  1998). 

■ Work  began  on  partner  selection,  model  development,  role 

descriptions,  communications  planning,  recruitment  and  training 
processes.  Three  community  agencies  were  selected  as  program 
delivery  sites  (Bent  Arrow  Traditional  Healing  Society,  Norwood  , 

Child  and  Family  Resource  Centre,  and  Terra  Association). 

Contracts  were  formalized  with  the  three  agencies  (October  1998). 
Staff  were  hired  and  trained  by  certified  trainers  from  the  Healthy 
Families  America  program.  Program  sites  began  accepting  referrals 
(November  1998). 

■ Program  support  and  leadership  for  front-line  staff  is  provided  by  j 

Team  Leaders,  one  at  each  of  the  three  program  sites. 

■ The  program  has  involved  ongoing  development  of  curriculum  and 
program  materials  (this  was  not  provided  by  the  Healthy  Families 
America  program). 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators  1 

Background  work  completed/existing  commitment: 

■ Significant  progress  had  been  made  in  advance  of  HTF  funding 
(CHA  support/interest,  literature  review,  attendance  at  conference, 
committees  in  place,  etc.)  which  facilitated  project  implementation 
(the  program  was  running  within  four  months  of  receiving  funding). 

Partnerships  and  Leadership 

■ Partnerships  were  well  established,  leadership  present. 
Evidence-based  implementation 

■ The  Healthy  Families  Program  was  modeled  on  key  elements 
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identified  in  the  literature  (i.e.,  the  program  must  be  intensive, 
comprehensive,  long-term  in  its  involvement  with  families,  culturally 
appropriate  and  provide  services  in  the  identified  outcome  areas). 

Challenges 

Short  timelines 

■ Because  of  short  timelines  (from  conceptualization  to 
implementation)  there  were  some  areas  that  received  less  attention 
than  might  otherwise  have  been  the  case.  For  example,  program 
promotional  materials  were  not  available  until  well  into  program 
implementation.  More  planning  time  may  have  resulted  in  the 
earlier  development  of  both  (a)  curriculum  and  tools  for  Family 
Visitors  to  use  with  families  and  (b)  an  information  management 
database. 

Lack  of  understanding  by  other  agencies 

■ Lack  of  understanding  by  other  agencies  contributed,  initially,  to  a 
slow  rate  of  referrals.  Early  work  was  necessary  to  communicate 
the  intent  and  scope  of  the  project  to  some  service  agencies  and 
organizations  (e.g.,  to  clarify  that  the  Healthy  Families  program  was 
intended  to  be  part  of  the  continuum  of  services  to  families,  not  to 
take  over  the  roles  of  other  related  agencies).  This  appears  to  have 
been  successful,  in  that  the  program  staff  and  management  felt  that 
the  program  was  well  accepted  as  part  of  the  host  programs  offered 
within  the  community. 

Issues  identified  by  independent  evaluator 

■ Confusion  around  evaluators’  roles  (HTF  and  Justice  Canada). 

■ Lack  of  database  for  monitoring. 

■ Late  start  in  collecting  outcomes  data  (6-month  delay  for  baseline). 

■ Challenges  in  collecting  outcomes  data  (lengthy  instruments, 
resistance  from  staff). 

■ Impact  of  evaluation  duration  (evaluators  modified  initial  workplan 
when  HTF  timelines  were  extended  - a third  round  of  interviews 
and  site  visits  were  included). 

Results 

Limitations 

Independent  evaluators  indicated  two  major  limitations  to  the 
interpretation  of  results: 

■ Families  enrolled  in  the  program  face  significant  barriers.  It  may  be 
more  important  to  address  how  they  have  improved  versus  how 
they  compare  to  the  general  population. 

■ Outcome  measurement  instruments  were  not  implemented  until 
June  1999  minimizing  measurement  of  change  between  Time  1 
and  Time  2. 

Participants 

■ Over  the  life  of  the  program,  389  families  have  been  positively 
screened,  344  families  referred,  and  248  enrolled  in  the  program. 
Of  the  248  families  enrolled  89  have  dropped  out  of  the  program. 

■ Family  Visitors  have  provided,  on  average,  2 visits  per  month  per 
family  or  2.87  hours  per  family  per  month. 

1.  Parents  will  have  increased  knowledge  and  use  of  community 
supports. 

■ Based  on  pre-post  measures  of  social  support  based  on 
responses  (N=21)  to  the  Maternal  Social  Support  Inventory 
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(MSSI),  most  parents  did  not  show  an  increase  in  their  social 
supports. 

■ Capital  Health  data  indicated  that  at  the  time  of  birth  between 
81%  - 90%  of  mothers  (at  the  three  agencies)  identified  a 
personal  physician. 

■ Healthy  Families  infants  are  nearly  twice  as  likely  to  visit  the 
emergency  room  as  are  other  Capital  Health  infants.  There  was 
a significant  decrease  in  the  use  of  Emergency  Departments 
over  the  mother’s  time  of  involvement  with  Healthy  Families. 

■ Clients  value  the  important  connections  that  have  been 
provided  to  them  by  the  agency.  Agencies  are  using  a number 
of  mechanisms  such  as  direct  referrals  to  other  agency 
programs  offering  daycare,  group  activities,  and  introduction  to 
other  staff  members. 

■ Approximately  70%  of  families  were  functioning  in  the  normal 
range  based  on  the  Family  Assessment  Device  (FAD).  Pre-post 
measures  of  family  functioning  revealed  less  positive  family 
functioning  over  time  (N=23).  (It  may  be  important  to  consider 
sample  size  and  increased  strain  as  child  progresses  in  age.) 

2.  Parents  will  improve  in  their  personal  development. 

■ There  is  little  direct  data  to  address  whether  or  not  Healthy 
Families  parents  have  improved  in  their  ability  to  set  and  meet 
personal  goals. 

■ The  majority  of  families  surveyed  found  the  goal-setting 
process  "always"  or  "almost  always  helpful"  and  indicated  that 
they  felt  they  were  a more  capable  person  because  of  their 
relationship  with  their  Family  Visitor. 

■ Written  goal  plans  were  recorded  for  56%  to  80%  of  clients  at 
the  three  agencies. 

■ Parents  scored  high  on  a scale  measuring  parental  knowledge 
of  four  key  areas  of  child  development:  emotional,  cognitive, 
physical,  and  social.  Scores  ranged  from  73  to  92  (maximum 
score  100).  Knowledge  of  social  development  was  the  least 
developed  but  improved  over  time. 

■ There  have  been  some  subsequent  pregnancies  among 
Healthy  Families  mothers. 

3.  Children  will  have  healthy  growth  and  development. 

■ 10.8%  of  mothers  in  the  Healthy  Families  Program  had  a low 
birth  weight  baby.  The  average  for  the  Capital  Health  Region  is 
6.4%.  This  represented  12%  Bent  Arrow  mothers,  20% 

Norwood  mothers,  and  only  1.8%  Terra.  Of  the  three  sites, 

Terra  has  the  largest  proportion  of  mothers  who  enter  the 
program  prenatally. 

■ At  the  point  of  leaving  the  hospital,  between  59%  and  80%  of 
mothers  were  breast-feeding  their  babies.  The  region-wide  rate 
is  73%. 

■ As  of  March  31 , 2000,  89-90%  of  babies  were  up  to  date  with 
their  immunizations.  The  Capital  Health  average  is  94%  at  the 
age  of  one  year.  Anecdotal  evidence  from  families  and  Family 

■ Visitors  suggests  the  rate  of  immunization  would  have  been 
much  lower  for  many  of  the  families  without  their  involvement  in 

the  Healthy  Families  program. 
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■ The  Denver  Developmental  Screening  Test  II  (DDS-II)  was 
used  to  monitor  healthy  growth  and  development.  Results 
based  on  the  testing  of  84  children  indicated  that  80%  were 
within  the  normal  range  for  development.  Of  the  22  children 
who  received  the  second  test  for  development,  91%  were 
within  the  normal  range  (an  increase  from  80%  at  baseline). 
(These  data  should  be  interpreted  with  caution  since  if  delays 
are  going  to  occur  or  worsen  they  are  more  likely  to  occur  in 
the  2-3  year  old  age  group.) 

■ The  majority  of  parents  responding  to  satisfaction  survey  (1 9 
out  of  22)  agreed  that  their  child’s  development  has  been 
improved  because  of  the  families’  involvement  with  the  Family 
Visitor. 

■ Mothers  involved  in  the  Healthy  Families  Program  were  more 
likely  to  use  cigarettes,  alcohol,  or  street  drugs  during 
pregnancy  than  the  overall  Capital  Health  population. 

4.  Parents  will  develop  positive  parent/child  interactions. 

■ The  Adult-Adolescent  Parenting  Inventory  II  (AAPI)  was  used 
to  measure  parenting  skills  according  to  five  key  attributes 
(parental  expectations,  empathy  for  and  awareness  of  the 
child’s  needs,  definition  of  child  and  parental  roles,  child’s 
power  and  independence,  and  belief  in  corporal  punishment). 

Of  the  71  parents  who  completed  the  scale  initially,  most 
scored  average  or  high  on  the  first  three  attributes.  Of  those 
who  responded  a second  time  (N=23),  there  were  fewer 
parents  who  believed  in  corporal  punishment  as  an  appropriate 
form  of  discipline. 

■ There  have  been  55  Child  Welfare  investigations  in  the  Healthy 
Families  caseload.  Of  these,  28  were  unfounded,  2 
substantiated  for  abuse,  and  25  substantiated  for  neglect. 

■ 4 mothers  have  had  previously  apprehended  children  returned 
to  their  care  as  a direct  result  of  the  support  and  parenting 
knowledge  gained  through  the  Healthy  Families  program. 

■ The  majority  of  parents  responding  to  the  client  survey  reported 
that  their  relationship  with  their  child  is  better  because  of  the 
presence  of  the  Family  Visitor  in  their  lives. 

Other  Impacts 

Healthy  Families  Staff 

■ Staff  reported  a great  deal  of  support  from  the  community 
agencies  and  have  benefited  from  a vast  amount  and  variety  of 
training.  However,  turnover  has  been  an  issue  with  some  staff 
indicating  that  salary  levels  were  too  low. 

Community  Agencies 

■ The  agencies  selected  for  participation  were  seen  as  a good  fit 
for  the  Healthy  Families  model. 

Capital  Health 

■ There  has  been  support  for  the  program  within  Capital  Health 
with  other  areas  of  the  region  requesting  a similar  program 
within  their  catchment  areas  to  which  they  can  refer  at-risk 

families.  The  challenge  is  to  secure  funding. 
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Learning 

Issues  identified  by  project  team 

■ Importance  of  involving  at-risk  mothers  prenatally  to  counteract 
high-risk  pregnancy  behaviours  (evidenced  by  Terra  mothers’  lower 
incidence  of  pre-term  and  low  birth  weight  babies). 

■ Importance  of  leadership  and  commitment  to  the  program  (program 
champions). 

■ Importance  of  community-based  implementation  and  program 
delivery. 

■ Importance  of  existing  and  mature  community  agencies  that  offer  a 
wide  variety  of  services. 

■ Importance  of  longer  timeframes  for  outcome  evaluations. 

■ Variations  from  the  Healthy  Families  America  program  were 
required  such  as  varied  enrollment  criteria,  focus  on  family  support 
versus  child  abuse  prevention  (labeling),  and  the  creation  of  a 
curriculum  package  for  Family  Visitors. 

■ Communication  with  physicians  requires  more  attention  (i.e., 
communicating  the  program  scope  and  intent,  how  to  refer  families 
and  who  to  refer  to). 

Current  Challenges 

■ Funding  - achieving  commitment  from  funders  is  a major  issue. 

■ Information  Management  System  - in  progress/development 

■ Child  Welfare  Issues  - continue  to  improve  relationship  with 
Ma’Mowe  Child  and  Family  Services. 

■ Subsequent  Pregnancies  - Caseload  weighting  system  may  need 
readjusting  to  reflect  additional  time  required  to  work  with  families 
with  more  than  one  child. 

■ Staff  T urnover  - salary  levels  need  to  be  re-examined. 

■ Expansion  of  the  Program  - need  set  of  criteria  to  select  sites. 

■ Capacity  to  Support  Culturally  Diverse  Families  - consider 
including  Family  Visitors  from  more  diverse  ethnic/language 
backgrounds. 

■ Accreditation  - program  management  is  investigating  avenues  for 
accreditation. 

■ Need  to  achieve  a higher  level  of  physician  involvement. 

Contribution  to  National  Dimensions 

Access 

■ 

■ 

■ 

At-risk  families  are  supported  in  accessing  a wide  range  of  health 
services  that  they  may  not  have  accessed  independently.  For  example, 
available  services  include  postpartum  depression  screening,  referrals  to 
mental  health  services,  occupational  therapy  services,  assistance  in 
finding  a “medical  home”  (family  practitioner,  etc.),  and  linkages  to 
Ma’Mowe  Child  and  Family  Services  through  a liaison  worker  assigned 
to  each  of  the  three  delivery  sites. 

There  are  many  routes  and  many  opportunities  for  those  families  who 
are  identified  as  at-risk  to  gain  access  to  the  Healthy  Families  Program 
(e.g.,  screening  by  Public  Health  Nurses,  referrals  from  any  other  health 
service  or  health  service  provider,  from  community  agencies,  or  self- 
referrals). 

The  community  agency  setting  provides  a range  of  programs  in  the 
community.  The  nurse  who  is  part  of  the  Healthy  Families  team  at  one  of 
the  sites  has  offered  immunization  clinics  and  well  baby  clinics  at  the 
site. 

Transportation  support  is  often  offered  to  clients  to  ensure  families  are 
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able  to  access  services  they  require. 

Quality 

■ Almost  all  families  involved  in  the  program  have  expressed  a high  level 
of  satisfaction  with  the  services  provided.  Staff  and  stakeholders  also 
express  high  levels  of  satisfaction. 

■ Family  Visitors  provide  encouragement,  support,  reminders. 

■ Intensive  staff  training  (initially)  and  ongoing  training  (currently  once  per 
month)  contributes  to  the  quality  of  the  program. 

■ Local  trainers  are  certified.  A train  the  trainer  approach  allows  local 
trainers  to  ensure  training  is  complete  and  appropriate  for  the  local 
context. 

Integration 

■ The  collaborative  partnership  approach  to  the  development  of  the 
Program  has  helped  to  ensure  an  integrated  support  system  for  families 
enrolled  in  the  Program. 

■ The  involvement  of  inter-sectoral  and  multidisciplinary  representatives 
on  the  committees  helps  support  the  delivery  of  integrated  services  to 
families. 

■ The  integration  of  the  Program  into  existing  community-based  services 
and  being  embedded  in  existing  community  agencies,  facilitates  access 
by  families  to  a wide  variety  of  social  and  support  services,  as  well  as 
health-related  services.  This  collaboration  with,  and  building  onto 
existing  service  delivery  systems,  also  increases  capacity  within  that 
community  agency  system. 

Health  Status 

■ The  Healthy  Families  program  has  the  potential  to  positively  impact  the 
health  of  the  families  over  the  long  term.  More  time  is  required  before 
definitive  statements  can  be  made.  Some  preliminary  findings  include 
fewer  pre-term  and  low  birth  weight  babies  among  mothers  from  the 
Terra  organization;  breast-feeding  rates  between  59%  and  80%  of 
mothers;  immunizations  up-to-date  for  between  89-90%  of  babies;  80% 
children  within  the  normal  range  for  development. 

■ The  Program  facilitates  up-to-date  immunizations  for  the  children  and 
birth  control  planning,  often  by  accompanying  the  family  to  the  Health 
Centre. 

Cost- 

Effectiveness 

■ Longer  term  evaluations  conducted  by  the  American  Healthy  Families 
programs  cite  a $6  to  $3  payback,  fewer  child  welfare  cases,  prevention 
of  health  problems,  expanded  social  network  for  parents,  improved  child 
development  and  readiness  for  school.  The  18-month  timeframe  for  this 
evaluation  precludes  measurement  of  cost-effectiveness  (3  to  5 years 
are  required). 

Transferability 

■ The  independent  evaluators  concluded  that  the  Capital  Health  region’s 
model  is  only  possible  where  there  is  a mature  community  agency  base 
offering  a range  and  variety  of  services  to  families  and  where  true 
partnerships  are  understood  and  practiced.  They  recommend  that  if  the 
program  is  to  expand  within  CHA,  the  agencies  be  handpicked,  as  there 
is  a large  amount  of  time  and  commitment  required  from  those  in 
management  positions  within  a respective  agency  (critical  to  the  success 
of  the  program). 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

Healthy  Families  incorporates  the  principles  of  primary  health  care  by 
supporting  families  with  social,  economic,  and  cultural  barriers  to  develop  the 
skills  and  capacity  to  promote  a healthier  future  for  their  children.  The 
Program  philosophy  is  consistent  with  a broad  definition  of  health.  At  least 
three  sites  in  Alberta  (Capital,  WestView,  and  Hobbema)  and  two  across 
Canada  (Yukon  and  Charlottetown)  have  implemented  the  Healthv  Families  | 
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Program.  j 

The  two  Healthy  Families  Programs  funded  by  the  HTF  have  been 
implemented  in  significantly  different  regions.  Three  primary  differences  are 
(1)  the  urban/rural  nature  of  the  regions;  (2)  the  mix  of  community  agencies 
delivering  services;  and  (3)  the  degree  of  community  partnerships  in  the 
delivery  of  the  program.  Comparison  of  results  across  these  different 
contexts  is  important  to  determine  how  the  program  should  be  tailored  to 
meet  the  needs  of  specific  groups. 

Funding  policies/priorities 

■ Funding  remains  a major  issue.  Need  for  long-term  funding  commitment. 
Screening  for  pregnant  women 

• Need  for  increased  prenatal  referrals  (opportunity  for  increased 
physician  involvement) 

Intersectoral  Partnerships/Physician  Involvement 

■ There  is  a continuing  need  to  ensure  an  effective  working  relationship 

between  the  Program  and  Child  Welfare.  Parents  who  have  had  children 
returned  to  their  care,  as  well  as  apprehensions  that  have  been  jj 

prevented  by  Healthy  Families  involvement,  represent  a significant 
savings  to  the  Child  Welfare  system. 

■ There  has  been  successful  demonstration  of  program  embedded  into  the 
community  infrastructure  rather  than  operated  by  the  health  authority. 
Agencies  may  need  to  be  purposefully  selected  to  ensure  they  practice  a 
family-centred  approach  to  care. 

■ Increased  linkages  to  local  physicians  would  provide  an  excellent  source 
of  advice  and  information  on  health-related  issues,  could  support  an 
increase  in  the  number  of  families  having  access  to  a family  physician, 
and  may  assist  in  achieving  increased  numbers  of  prenatal  referrals. 

Personnel  policies 

■ Issues  of  short-term  staff  in  a long-term  program  and  salary  levels  need 
to  be  examined. 

Information  Systems 

■ An  Information  Management  System  is  required  to  facilitate  long-term 
study. 

Research 

■ Development  of  information  management  system  is  essential  for 
ongoing  collection  of  data  and  outcome  measurement. 

■ Information  gathering  (literature  review  conducted  and  key  informants) 
provides  an  opportunity  to  continue  building  on  the  existing  knowledge 
base. 

■ Opportunities  exist  for  collaborative  work  with  other  Healthy  Families 

Programs  across  Alberta  and  Canada.  ; 

Evaluation 

■ The  Healthy  Families  Program  creates  opportunities  for  further 
participatory  and  collaborative  evaluations  (e.g.,  with  Justice  Canada). 

■ There  are  multiple  benefits  of  information  sharing  (coordination, 
efficiency,  value). 

■ Need  to  minimize  burden  of  instrumentation/measurement  while 

maintaining  accurate  long-term  outcome  data.  ; 
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35 


Synopsis  of  Independent  Evaluation  Reports 


Project  Name 

Healthy  Families  - WestView 

(Demonstration  Project) 

Project  Description 

The  WestView  Healthy  Families  Program  is  modeled  after  the  Healthy 
Families  America  Program.  The  program  targets  families  having  their 
first  child  and  provides  intensive  home  visiting  provided  by  a trained 
Family  Visitor. 

Expected  Outcomes 

For  Children  and  Parents 

1.  Promote  positive  parent-child  interaction. 

1.1  To  increase  parents’  knowledge  and  understanding  of  their 
child’s  healthy  development. 

1.2  To  increase  the  use  of  positive  strategies  in  child  rearing  and 
behaviour  management,  and  decrease  child  abuse/neglect  and 
use  of  corporal  punishment. 

2.  Holistic  healthy  child  development. 

2.1  To  ensure  healthy  physical  development  of  the  child. 

2.2  To  ensure  healthy  development  of  the  child’s  cognitive,  motor, 
social,  and  cultural  skills. 

3.  Support  parents’  functional  development. 

3.1  To  support  parents’  ability  to  identify  and  make  decisions  about 
family,  social,  health,  and  personal  needs/goals. 

3.2  To  support  parents’  ability  to  develop  skills  and  confidence  in 
managing  their  own  lives  and  their  households,  while  taking 
care  of  their  child. 

4.  Increase  knowledge,  competence  in  using,  and  use  of  community 

supports. 

4.1  To  increase  parents’  awareness  and  knowledge  of  family 
supports  available  in  their  community,  prenatally  and  in  the  first 
12  months  of  life. 

4.2  To  increase  parents’  comfort  and  confidence  in  accessing 
community  support  services  and  health  resources. 

4.3  To  increase  parents’  participation  in/access  to  community 
support  resources. 

4.4  To  increase  family  compliance  with  well  baby  clinics  and/or 
pediatrician  follow  up,  immunization  and  other  identified  health 
visits  throughout  the  child’s  first  year. 

4.5  To  promote  communication  and  co-operation  between 
community  agencies  in  supporting  families  in  the  program. 

For  the  Proaram 

Specific  outcomes  were  also  identified  for  program  development,  staff 
training,  program  entry,  recruitment  of  participants,  family  visits,  case 
management  to  ensure  that  the  Family  Visitor  maintains  perspective  on 
the  families  and  on  their  own  needs,  and  to  provide  supervisory  support 
to  the  Family  Visitor  as  well  as  to  decide  whether  a change  of 
frequency  and/or  program  change  is  appropriate. 

Methodology  of 

Independent 

Evaluation 

■ Review  of  client  records 

■ Review  of  program  records 

■ Development  of  Individual  Family  Information  Form  to  collect  family 
data 

■ Denver  Developmental  Screening  Test  (DDST) 

■ Site  visits/observation 

■ Client  survey  (telephone  interviews) 

■ Family  Visitor  and  Public  Health  Nurse  focus  groups 

■ Review  of  program  process 

Healthy  Families  - WestView 

(Demonstration  Project) 
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Review  of 

Implementation 

Process 

The  WestView  project  secured  funding  through  a partnership  between 

West  Yellowhead  Family  and  Children’s  Services  Authority  and  the 

WestView  Regional  Health  Authority. 

■ Initial  recruitment  and  training  of  staff  occurred  in  Sept/Oct  1998. 
Client  recruitment  began  in  December  1998. 

■ Seven  (7)  Family  Visitors  and  fifteen  (15)  Public  Health  Nurses 
participated  in  a four-day  training  session  facilitated  by  certified 
Healthy  Families  America  trainers  in  Edmonton.  A second  training 
session  was  held  for  new  recruits  in  June  1999. 

■ The  WestView  Healthy  Families  program  operates  from  six  Health 
Units  in  the  region  (Devon,  Stony  Plain/Spruce  Grove,  Evansburg, 
Edson,  Hinton,  and  Jasper). 

■ A Regional  Steering  Committee  and  Communication  Plan  were 
developed. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

Strong  association  of  the  WestView  Healthy  Families  Program  with  the 

Public  Health  Nursing  program  is  a strength  of  the  Program. 

Challenges 

■ The  program  coordinator  position  was  initially  part  time  (0.5  FTE) 
and  the  individual  did  not  have  responsibility  for  the  day-to-day 
supervision  and  management  of  staff.  This  proved  challenging  and 
was  changed  to  a full-time  out-of-scope  manager  position  with 
direct  supervision  and  management  responsibilities. 

■ Lower  than  expected  client  enrollment  rates  (after  8 months  of 
implementation  there  were  19  families  enrolled;  after  16  months 
there  were  45  families) 

■ Some  Public  Health  Nurses  (PHN)  indicated  that  they  were  carrying 
out  Healthy  Families  responsibilities  in  addition  to  their  regular 
duties.  In  response  to  the  workload  of  PHNs,  the  program 
coordinator  dedicated  one-third  of  her  time  to  the  sites  to  provide 
training/support;  an  automated  database  is  being  developed  to 
reduce  the  amount  of  manual  data  gathering;  a supervision  model 
is  in  development;  and  workload  issues  will  be  closely  monitored  by 
the  program  coordinator. 

■ Role  confusion  existed  around  the  regional  Steering  Committee. 
The  terms  of  reference  for  the  committee  indicated  it  was  to  provide 
direction  in  the  implementation  of  the  program.  However,  by  the 
time  the  committee  had  its  first  meeting,  program  implementation 
was  well  underway  and  key  decisions  about  its  implementation  had 
been  made.  Moreover,  it  has  since  been  indicated  that  little 
direction  regarding  implementation  was  necessary  since  the 
Program  was  modeled  after  an  existing  program  (Healthy  Families 
America)  and  that  the  term  “steering”  was  a misnomer.  The 
committee  disbanded  at  project  completion  (May  2000). 

Results 

■ Client  statistics  from  December  1998  to  March  2000  indicated  that 
45  clients  were  enrolled  in  the  Program.  Originally  75  families  were 
expected  in  the  program  although  the  numbers  have  been  lower.  It 
is  suggested  that  this  may  be  because  of  the  two-week  eligibility 
criteria.  Also  many  parents  already  had  well-established  social 
support  networks  and  all  had  access  to  a family  physician. 

37 


Synopsis  of  Independent  Evaluation  Reports 


Specific  outcomes  included 

Client  Satisfaction 

■ Overall,  90%  of  the  families  rated  the  Program  as  "excellent"  or 
"good."  Satisfaction  was  higher  when  the  number  of  in-home  visit 
hours  was  higher  (more  than  21  hours),  the  baby  was  over  a year 
old,  the  baby  was  premature/low  birth  weight  and/or  there  were 
developmental  or  neglect  issues,  and  where  the  mothers  were  over 
22  years  of  age.  Satisfaction  was  lower  when  the  mother  lived  with 
her  parents. 

Other 

■ The  Family  Visitor  and  Public  Health  Nurse  compiled  a list  of 
positive  parenting  and  health  enhancing  techniques  practiced  by 
parents  (N=61)  recorded  on  the  Individual  Family  Information  Form. 
The  most  frequently  cited  positive  techniques  included  stimulating 
motor  development,  stimulating  social/emotional/cognitive 
development,  and  encouraging  bonding/attachment.  Also,  85%  of 
mothers  correctly  identified  the  age  group  for  eight  of  ten 
developmental  milestones.  Almost  half  indicated  that  they  had 
learned  most  of  what  they  knew  about  their  child’s  healthy 
development  from  their  Family  Visitor.  Those  less  likely  to  attribute 
this  knowledge  to  the  Family  Visitor  lived  with  a parent. 

■ Family  Visitors  reported  that  60%  of  the  families  used  positive 
parenting  strategies  on  a regular  basis.  The  target  set  for  the 
evaluation  was  75%;  thus  it  was  suggested  that  teaching  strategies 
in  relation  to  positive  discipline  techniques  be  enhanced.  Only  one 
child  (out  of  61)  was  removed  from  the  home  by  Child  Protection 
Services  following  a report  by  the  program  staff  of  suspected 
neglect. 

■ The  Denver  Development  Screening  Test  (DDST)  was  used  to 
assess  each  child’s  growth  and  development.  97%  of  those  tested 
were  within  the  normal  range  for  growth  and  development  as  well 
as  cognitive,  motor,  social,  and  cultural  skills. 

■ On  average,  75%  of  parents  thought  they  had  been  able  to  make  a 
change  they  wanted  (such  as  looking  at  options  for  their  own  life, 
finding  ways  of  getting  financial  support).  Most  parents  agreed  that 
their  Family  Visitor  had  encouraged  them  to  figure  out  what  they 
wanted  to  do  or  how  they  should  do  it.  Family  Visitors  reported  that 
about  73%  of  the  goals  had  either  been  achieved  by  families  (57%) 
or  were  being  worked  on  (16%).  About  28%  of  mothers  under  the 
age  of  21  returned  to  school  within  6 months  of  having  their  babies 
(less  than  the  target  of  85%  suggesting  that  flexible  school  options, 
adequate  financial  support  and  accessible  childcare  may  require 
further  attention). 

■ The  majority  of  parents  reported  that  the  Program  had  a role  in 
boosting  their  confidence  in  managing.  Also,  90%  of  families  who 
had  been  in  the  Program  for  three  months  had  completed  a parent 
action  plan  (in  keeping  with  the  recommended  timeframe  in  the 
HFA  Program  Manual). 

■ Overall,  86%  of  parents  were  able  to  name  at  least  one  community 
agency.  At  least  83%  of  all  parents  had  contacted  at  least  one  other 
agency  besides  the  Healthy  Families  Program.  Overall,  91%  of 
clients  thought  the  Program  had  helped  them  become  more 
confident  in  making  community  contacts.  Fifty-four  percent  (54%) 
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indicated  that  they  had  found  at  least  one  community  or  health 
service  on  their  own.  Eighty-six  percent  (86%)  of  families  had  used 
at  least  one  service  besides  the  Healthy  Families  Program.  Fifty- 
eight  percent  (58%)  of  families  were  satisfied  with  the  referrals 
made  to  community  agencies.  Overall,  70.5%  of  children  were 
immunized  on  time;  80%  had  their  immunizations.  All  families 
(100%)  had  identified  a family  physician.  Information  was  not 
collected  on  emergency  room  visits.  The  majority  (78%)  of  parents 
kept  their  well  baby  clinic  appointments  at  the  Health  Unit. 

Results  of  other  components  of  Program  implementation 

Program  Development 

■ All  sites  used  the  same  Program  forms.  The  program  coordinator 
was  instrumental  in  ensuring  program  consistency  and  took  a 
leadership  role  in  developing  and  maintaining  program  policies, 
standards,  guidelines,  and  procedures.  This  was  accomplished  by 
regular  meetings  with  program  staff,  review  of  the  program 
documentation,  and  dedicated  in-service  sessions. 

Staff  Training 

■ Staff  training  and  ongoing  educational  opportunities  were  a priority 
for  the  program  management  team  including  the  program 
coordinator.  Staff  completed  core  HFA  training  as  well  as  monthly 
educational  sessions.  Each  Family  Visitor  received  an  average  of 
14  hours  training  per  month.  Further  work  is  required  on  a 
supervisory  model  to  ensure  consistency  and  understanding  of  the 
role  of  the  supervisor.  (Public  Health  Nurses  were  assigned 
supervisory  responsibilities  for  the  Family  Visitors.) 

Program  Entry 

■ All  clients  in  the  Program  met  the  eligibility  criteria  (as  defined  by 
the  HFA  program  manual).  Program  staff  achieved  a 72% 
completion  rate  of  in-depth  assessments  when  the  Record  Screen 
was  positive. 

Family  Visitor  and  Client 

■ Relationship  development  activities  were  being  carried  out 
effectively  (e.g.,  97%  of  clients  felt  the  Family  Visitor  was  there 
when  needed).  Families  identified  and  developed  goals.  Most 
parents  felt  the  Family  Visitor  provided  extra  information,  and  many 
(67%)  had  used  most  of  the  information.  Overall,  97%  indicated 
they  had  been  assisted  by  the  program  in  meeting  family  goals. 
Parents  were  able  to  devise  plans  to  cope  with  stressful  situations 
in  more  than  90%  of  cases.  The  majority  of  parents  (91%)  felt 
comfortable  providing  feedback  to  their  Family  Visitors. 

Case  Management 

■ Overall,  the  Family  Visitors  and  Public  Health  Nurses  reported  the 
supervision  session  as  useful.  As  noted  previously,  further  work  is 
required  to  develop  a supervisory  model  to  ensure  consistency  and 
understanding  of  the  role  of  the  supervisory  and  the  expectations  of 
the  Family  Visitor. 

Learning 

■ It  is  important  to  clearly  define  roles  and  expectations  of  project 
teams,  evaluators,  and  steering  committees. 
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■ A program  coordinator  was  required  to  manage  the  region-wide 
Program. 

■ Staff  training  costs  were  substantial.  There  was  a recognized  need 
to  build  training  capacity  internally. 

■ Ongoing  education  of  Family  Visitors  continues  to  be  important. 

■ The  supervision  of  Family  Visitors  and  support  to  supervisors  is 
important. 

■ There  were  time  pressures  for  some  Public  Health  Nurses  involved 
in  the  program. 

■ Recruitment  of  clients  in  the  program  should  be  expected  to  be 
lower  than  desired  especially  at  the  onset  of  the  program. 

■ More  than  50%  positive  assessment  rate  of  families  screened  for 
the  program. 

■ Program  participant  numbers  were  lower  than  anticipated  (45 
families  Mar/00). 

■ The  regional  Steering  Committee  was  not  necessarily  steering  or 
leading  the  direction  of  the  Program. 

■ Documentation  and  quality  control  are  important. 

Contribution  to  National  Dimensions 

Access 

Access  to  the  WestView  Healthv  Families  Proqram 

■ The  Healthy  Families  Program  provided  access  to  intensive  home 
visitation  support  for  new  mothers. 

■ Access  may  be  increased  by  improving  program  marketing  strategies, 
since  follow-up  interviews  revealed  that  72%  of  families  who  refused  the 
Program  could  not  recall  its  general  purpose. 

Access  to  other  communitv  suDDorts 

■ The  Program  had  a positive  impact  on  a family’s  ability  to  access  the 
programs/services  they  needed.  The  majority  of  clients  were  able  to 
name  at  least  one  other  community  support,  had  contacted  at  least  one 
other  agency  (e.g.,  some  attended  Well  Baby  clinics  at  the  Health  Unit). 

■ Family  Visitors  played  an  important  role  in  helping  the  families  connect 
with  other  services.  They  also  fulfilled  a brokering  role  between  the 
family  and  the  community  agencies. 

Quality 

■ Quality  of  service  was  enhanced  through  intensive  training  requirements 
(initially  and  ongoing)  of  staff,  supervision  by  a Public  Health  Nurse,  and 
a case  management  model. 

■ The  program  reached  out  to  higher  needs  families  and  provided  them 
with  the  knowledge  and  skills  for  effective  parenting. 

Integration 

■ Having  the  WestView  Healthy  Families  Program  located  within  the 
Public  Health  Nursing  program  provided  continuity  between  the  Program 
and  other  programs  relevant  to  new  mothers  offered  by  the  Health  Unit 
(e.g.,  Well  Baby).  An  added  benefit  was  the  ongoing  support  that  the 
Public  Health  Nurses  provided  to  the  Family  Visitors,  especially  in 
relation  to  healthy  growth  and  development. 

■ While  one  area  for  improvement  was  marketing  of  the  program  to  other 
community  agencies/groups,  program  staff  connected  with  a wide  range 
of  community  agencies  when  assisting  families  to  make  other  community 
connections/referrals.  Referrals  were  made  to  a full  range  of  community 
agencies. 

Health  Status 

■ The  Program  demonstrated  the  importance  of  social  support  as  a 
determinant  of  health.  Building  rapport  between  the  family  and  the 
Family  Visitor  was  seen  as  an  essential  ingredient  for  success  of  the 
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program. 

■ Results  of  the  two-year  implementation  of  the  Healthy  Families  Program 
indicated  that  97%  of  children  were  within  the  normal  range  for  healthy 
physical  growth  and  development  as  well  as  cognitive,  motor,  social,  and 
cultural  skills. 

Cost- 

Effectiveness 

■ While  the  cost  per  family  per  year  was  estimated  at  approximately  $6500 
(Canadian),  which  is  comparable  with  similar  programs  delivered  under 
HFA  in  the  United,  the  evaluators  concluded  it  was  not  possible  to 
estimate  the  cost-effectiveness  of  the  WestView  Healthy  Families 
Program  at  this  early  stage  of  its  implementation.  The  train-the-trainers 
approach  was  used  to  minimize  cost  implications  of  ongoing  training  and 
to  build  internal  capacity. 

Transferability 

■ Program  development  and  start-up  requires  a dedicated  program 
coordinator. 

■ The  training  models  of  the  HFA  approach  have  limitations  and  are  not 
reflective  of  the  Canadian  experience. 

■ The  train  the  trainers  approach  should  facilitate  transferability  of  the 
model,  particularly  locally.  The  WestView  staff  have  the  capacity  to  train 
new  Family  Visitors  (certified  HFA  trainers). 

■ Ongoing  education  of  the  Family  Visitors  is  an  important  aspect  of  the 
program. 

■ Supervisors  require  ongoing  support  to  fulfill  the  role  of  supervising  the 
Family  Visitor. 

■ In  considering  transferability  it  is  important  to  consider  publicly  funded 
versus  private  health  systems.  All  families  in  the  WestView  program 
identified  a family  physician.  They  had  full  access  to  health  unit 
programs  and  other  community-based  support  services  unlike  families  in 
the  U.S.  upon  which  the  HFA  program  was  based. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

Proaram  Delivery  Sites  s 

■ In  comparing  the  Capital  Region  and  WestView  Health  Families  reports, 
the  issue  of  where  Healthy  Families  Programs  should  be  operated  is  not 
resolved.  Capital  included  three  community-based  agencies.  WestView 
included  three  Public  Health  Units.  Some  members  of  the  regional 
steering  committee  (WestView)  expressed  a preference  for  the  service 
delivery  model  in  Edmonton.  The  strong  association  of  the  WestView 
Healthy  Families  Program  with  the  Public  Health  Nursing  program  was 
seen  as  a strength  overall  of  the  WestView  Program. 

Points  for  further  discussion 

■ What  are  the  most  appropriate  sites  for  program  delivery?  Health  units 
or  community-based  agencies  or  both? 

■ What  is  the  best  way  to  manage  overall  program  coordination?  What 
should  be  the  role  of  the  Program  Coordinator? 

■ What  is  the  most  effective  supervisory  model?  Public  Health  Nurse 
model  or  other? 

41 


Synopsis  of  Independent  Evaluation  Reports 


Implications  for 
Evaluation 


■ The  two-week  time  frame  to  complete  the  in-depth  assessment  was  too 
short;  it  should  be  extended  to  four  weeks. 

■ Initially  the  Healthy  Families  program  staff  focussed  on  program 

implementation  and  less  on  interagency  cooperation  and  collaboration. 
As  the  program  matured,  the  importance  of  interagency  collaboration 
became  increasingly  important.  The  independent  evaluators  indicated 
that  this  was  an  area  requiring  ongoing  commitment  by  all  program  staff 
and  should  be  flagged  as  an  area  for  continuous  improvement. 


42 


Synopsis  of  Independent  Evaluation  Reports 


Project  Name 

Keeweetinok  Lakes  Regional  Health  Authority  Immunization 
Project 

(Demonstration  Project) 

Project  Description 

The  Immunization  demonstration  project  was  conducted  in 
Keeweetinok  Lakes  Regional  Health  Authority — a geographically  large, 
culturally  diverse  (40%  Aboriginal)  and  sparsely  populated  region 
(approximately  25,000  people)  that  is  challenged  by  low  immunization 
rates.  The  project  evaluated  current  immunization  standards  and 
interventions  to  targeted  populations  at  high  risk  for  communicable 
diseases.  A chart  audit  identified  potential  participants  (those  with  low 
immunization  rates)  to  be  interviewed  in  their  homes.  Parents  were 
invited  to  sign  a refusal  form  or  make  a subsequent  appointment  for 
immunization.  Characteristics  and  barriers  towards  immunization  of  the 
target  population  were  examined. 

Expected  Outcomes 

1 . Maintenance  of  low  incidence  of  vaccine  preventable  diseases. 

2.  Increased  immunization  rate  of  children  0-3  years  of  age 
throughout  Region  15. 

3.  Increased  immunization  rate  of  children  0-3  years  of  age,  in 
communities  where  rates  are  lowest,  throughout  Region  15. 

4.  Strategies  used  to  previously  improve  immunization  rates  are 
evaluated. 

5.  Identification  of  characteristics  of  the  population  group  that 
demonstrates  reduced  immunization  compliance. 

6.  Identification  of  reasons  why  participants  in  the  target  population  do 
not  participate  in  immunization  as  per  Provincial  Immunization 
Standards. 

7.  Determination  of  the  effect  of  a refusal  request  on  subsequent 
adherence  to  immunization  schedules. 

8.  Development  of  an  affordable  and  sustainable  mechanism  to 
increase  immunization  rates  for  target  populations  with  lower  than 
provincial  standards  of  compliance. 

Methodology  of 

Independent 

Evaluation 

While  the  project  methodology  was  quantitative,  the  methods  selected 
for  evaluation  of  this  evaluation  project  were  almost  exclusively 
qualitative,  consisting  of  timely  reporting  of  project  activities  by  the 
KLRHA  Project  Coordinator,  monthly  interviews,  and  document  review. 

Review  of 

Implementation 

Process 

The  primary  thrust  of  the  project  design  was  to  first  establish  valid  and 
reliable  data  sets  that  could  subsequently  be  used  to  correlate 
immunization  rates  with  variables  thought  to  have  some  impact  on 
those  rates.  Data  set  development  entailed  a two-part  process: 

1 . Assurance  that  existing  data  were  valid  and  accurate. 

2.  Development  of  a tool  to  assist  in  identifying  characteristics 
(demographic  and  psycho-social)  of  clients  who  tended  to 
immunize  their  children  on  schedule,  immunize  but  did  not  adhere 
to  an  appropriate  schedule,  and  did  not  immunize. 

Phase  1 - Validation  of  Existing  Client  Database 
Validation  of  the  existing  data  was  implemented  to  establish  a valid 
frame  from  which  to  select  a sample  population  appropriate  for  the 
study.  Through  chart  audit,  examination  of  appropriate  immunization 
status  was  made  of  all  children  within  the  age  group.  Client  data  were 
compared  to  the  accuracy  of  data  contained  in  the  CHIIS  system. 

The  sample  population  was  divided  into  two  categories.  Using  the 
Alberta  Immunization  Standards  as  the  basis  for  determination,  two 
categories  of  subjects  were  selected:  the  first  category  children  who 

Keeweetinok  Lakes  Regional  Health  Authority  Immunization 
Project 

(Demonstration  Project) 
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had  received  age  appropriate  immunizations  and  were  up-to-date,  the 
second  category  children  who  were  under  immunized  according  to  age. 

Phase  II  - Survey  Design,  Development  and  Implementation 
Survey  design  was  based  on  an  existing  instrument  (second  draft  of 
Overdue  Immunization  Survey  obtained  from  the  Communicable 
Disease  Control/Immunization  Committee).  Survey  development  and 
validation  began  in  December  1998.  During  the  course  of  three  months 
the  survey  underwent  several  modifications  (10  versions)  and  was 
validated  using  multiple  methods.  Surveys  were  administered  in  person 
or  by  telephone  with  parents/guardians  of  children. 

Facilitators/ 
Challenges  to 
Implementation 

Challenges 

Several  issues  were  identified  that  impacted  data  collection.  Following 

is  a list  of  issues. 

1 . A substantial  amount  of  time  was  devoted  to  identify  and  correct 
data  errors  in  the  CHI  IS  database. 

2.  Selection  of  a sample  size  took  more  time  than  anticipated. 
Additional  numbers  of  participants  were  identified  resulting  in  the 
need  for  additional  resources  required  to  complete  the  study. 

3.  Development  of  an  appropriate  database  to  record  information 
derived  from  the  administration  of  instruments  was  slow. 

4.  Extension  to  the  interviewing  schedule  was  required.  Travel 
restrictions  (weather),  difficulty  locating  participants  contact 
numbers  and  addresses,  and  time  required  to  review  the  Child 
Health  Records,  contributed  to  a delay  in  start  up. 

5.  Changes  in  project  personnel  resulted  in  time  lost  and  delays  in 
getting  information  from  Alberta  Health  and  Wellness.  Each  delay 
involved  reworking  of  action  plans  and  budget  allocations. 

Confounders  to  Interpreting  Results 

Confounders  may  have  contributed  to  some  of  the  increase  in 

immunization.  While  these  confounders  may  have  positively  influenced 

the  increased  immunization  rates,  they  need  to  be  considered. 

1 . A Regional  procedure  was  developed  for  the  Overdue 
Immunization  Worksheet.  This  increased  the  overdue  immunization 
reminders  sent  out  in  some  parts  of  the  region.  These  extra 
reminders  may  have  influenced  the  under-immunized  group  to  get 
their  children  immunized. 

2.  The  process  of  getting  records  “in  shape”  may  have  stimulated 
some  nurses  to  pay  more  attention  to  under  immunization. 

3.  Throughout  the  project  there  was  ongoing  intervention  of  the 
nurses  performing  the  Overdue  Immunization  Follow  Up. 

4.  A fourth  confounder  may  have  had  a negative  effect  on  the 
immunization  rates.  Nursing  personnel  changed  in  many  areas  of 
the  region  during  the  course  of  the  study.  It  takes  time  for  the 
people  in  a given  area  (especially  in  small  communities)  to  trust 
and  feel  comfortable  with  new  faces. 

Results 

■ Four  hundred  sixteen  (416)  surveys  were  completed.  Participation 
rate  was  high  (66%)  with  only  21  refusals.  When  ineligibility  criteria 
were  considered  (moved,  unable  to  contact,  sibling  interviewed), 
response  rate  was  90.2%. 

■ The  project  team  reported  that  immunization  rates  of  the  UND 
(under-immunized)  group  living  in  rural  communities  had  increased. 
Thirty-seven  percent  (37%)  of  the  target  group  made  a change  in 
their  immunization  status  following  intervention  one  - Permission  to 
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Contact  Letter.  An  additional  26%  made  a change  after  the 
interview. 

The  Right  of  Refusal  was  only  applicable  and  offered  to  1 1 people. 
Written  refusal  did  not  appear  to  be  an  effective  strategy  in  this 
region  for  dealing  with  under-immunization. 

65%  (N=416)  of  survey  respondents  reported  that  the  current 
immunization  system  worked  for  their  family. 

A total  of  47.5  personnel  days  went  into  making  contact,  conducting 
the  interviews,  and  travel  for  this  study. 


Learning 


Reminder  systems  will  be  effective  in  increasing  immunization 
rates. 

The  resulted  suggest  that  primary  caregivers  of  the  under- 
immunized are  typically  single,  Aboriginal,  younger  in  age,  with 
lower  levels  of  education  and  income. 

Those  who  did  not  maintain  the  immunization  status  of  their 
children  appear  to  have  less  knowledge  about  immunization  and 
less  awareness  of  their  child’s  immunization  schedule. 

Statistically,  clinic  inaccessibility  proved  to  be  the  highest  predictor 
among  barriers  of  under-immunization.  Other  significant  barriers 
cited  by  the  under-immunized  group  were  lack  of  childcare  and 
transportation  problems. 

The  merit  of  personal  contact  with  clients  (telephone/face-to-face) 
cannot  be  over-emphasized  in  dealing  with  this  clientele. 
Non-compliant  subjects  demonstrated  a lack  of  persistence  in 
meeting  their  children’s  health  needs.  This  group  depends  heavily 
on  external  direction  from  health  professionals. 

The  team  emphasized  the  need  to  maintain  accurate  records, 
especially  contact  numbers  of  clients. 


Contribution  to  National  Dimensions 


Access 


The  primary  discussion  of  barriers  in  this  study  centred  on  identifying 
which  barriers  appeared  to  impede  access  to  immunization  services  by 
the  under-immunized  group.  This  study  revealed  that  while  clients  can 
identify  barriers,  they  are  much  less  adept  at  identifying  solutions. 
Approximately  78%  of  the  total  population  reported  barriers  to 
immunization.  Only  35%  who  offered  a solution  or  suggestion. 

The  importance  of  this  learning  is  that  while  barriers  may  exist,  clients’ 
perspectives  on  each  barrier  must  be  identified  before  the  assumption  is 
made  that  a particular  barrier  actually  impedes  access. 

The  approach  taken  by  the  project  had  a definite  impact  on  immunization 
rates.  The  “refusal  request”  element  was  shown  to  be  ineffective  and  has 
not  been  recommended  to  the  region  as  an  appropriate  strategy  to 
increase  immunization  rates.  The  team  suggested  that  the  region 
concentrate  on  improving  the  reminder/recall  system,  addressing  issues 
of  information  dissemination  to  clients,  and  reviewing  barriers  to  access 
to  immunization  services. 

Communication  seems  to  influence  the  motivation  of  caregivers  to 
access  and  utilize  immunization  services.  The  under-immunized  group 
tends  to  be  dependent  on  direct  encouragement  by  health  care  providers 
for  the  maintenance  of  their  health.  Under-immunized  groups  appear  to 
be  more  responsive  to  personal  communication  (phone  or  face  to  face) 
than  they  are  to  mail-out  requests  and  reminders.  In  providing 
reminders,  some  consideration  should  be  given  to  the  client’s  level  of 
understanding;  content  and  context  of  message  of  overdue  reminders; 
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content  of  messages  received  at  clinic  appointment;  timing  of 
information;  and  standardized  messages  to  promote  preventative  health 
behaviour.  A primary  health  care  model  that  promotes  individuals  to  take 
more  responsibility  for  their  own  health  will  require  the  implementation  of 
strategies  that  are  effective  in  weaning  clients  away  from  this 
dependency. 

Quality 

Integration 

Health  Status 

■ The  study  revealed  that  clients  labeled  as  under-immunized  have 

defining  characteristics  that  can  be  used  as  predictors  of  the  likelihood  of 
certain  primary  caregiver  groups  that  tend  not  to  immunize  their  children 
according  to  recommended  schedules,  if  at  all.  This  information  is  useful 
to  identify  groups  for  which  strategic  communication  is  expected  to 
improve  compliance  with  immunization  schedules.  Ultimately,  it  is 
expected  that  the  health  status  of  these  children  will  improve. 

Cost- 

Effectiveness 

■ The  sustainability  of  the  approach  used  in  this  study  will  depend  on  the 
region’s  response  to  the  following  questions: 

■ Was  the  52%  increase  in  immunizations  (approximately  20% 
increase  in  immunization  rates  across  the  region)  worth  the  47.5 
days  allocated  to  contact  and  interview? 

■ Was  the  decrease  in  identified  preventable  disease  worth  the  47.5 
days  allocated  to  contact  and  interview? 

■ This  study  has  provided  effectiveness  and  cost  data  that  can  be  used  for 
comparison  with  past  and  future  approaches.  These  comparisons  had 
not  been  completed  by  project  close. 

Transferability 

■ Care  and  attention  taken  in  choosing  the  appropriate  methodology  for 
this  project  allows  for  transferability  of  findings  to  contexts  similar  to  that 
of  Keeweetinok  Regional  Health  Authority. 

■ Instruments  underwent  reasonable  validity  checks.  However,  no 
reliability  calculations  were  conducted.  Future  iterations  of  the 
instruments  should  be  conducted  to  begin  building  both  reliable  and  valid 
assessments.  A detailed  report  of  the  project  team’s  experiences  and 
findings  (Van  Herk-Auger,  Denton,  Beyon,  and  Brockway,  2000)  is 
available  from  Alberta  and  Wellness  and/or  KLRHA.  Included  in  that 
report  are  several  appendices  outlining  the  approaches  and  instruments 
used. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 


Policy 

Implications 

Personal  reminders  have  a strong  likelihood  of  increasing  compliance  with 
immunization  schedules. 

Implications  for 
Research  and 
Evaluation 

Examination  of  information  in  existing  databases  should  be  conducted  to 
assess  the  currency  and  accuracy  of  immunization  data.  Since 
regionalization,  no  unified  mechanism  exists  at  the  provincial  level  to  assess 
overall  immunization  status  of  Alberta's  children. 
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Community  Outreach  in  Pediatrics/Psychiatry  and  Education 
Program 

(Research  Project) 


Project  Name 

Community  Outreach  in  Pediatrics/Psychiatry  and  Education 
Program 

(Research  Project) 

Project  Description 

The  Community  Outreach  in  Pediatrics/Psychiatry  and  Education 
(C.O.P.E.)  program,  a research  project,  investigated  earlier  and  more 
accurate  identification  and  diagnosis  of  children’s  mental  health 
problems  through  a school-based  intervention.  Medical  teams 
comprised  of  psychiatrists  and  pediatricians  provided  consultation 
related  to  diagnosis  of  children’s  mental  health  problems  in  four  study 
schools  which  were  randomly  selected  from  communities  within  the 
Calgary  Board  of  Education  known  to  have  a high  volume  of  need  and 
low  probability  of  access  to  system  resources.  High-risk  and  low-risk 
control  schools  were  also  included  in  the  study.  Medical  teams  worked 
collaboratively  with  school-based  education  teams  (Learning 
Specialists,  teachers,  administrators,  Family  Liaison).  There  was  no 
direct  cost  to  the  school  for  participation  in  C.O.P.E.  Physicians  billed 
for  consultation  time  according  to  instituted  fee-for-service  regulations 
established  for  the  province  of  Alberta. 

The  study  also  investigated  the  influence  of  building  cooperative 
relationships  between  medical  and  school-based  personnel  with 
respect  to  identifying  children’s  emotional,  behavioural  and  learning 
problems.  The  extent  to  which  families  were  more  knowledgeable 
about  their  children’s  mental  health  problems  including  the  extent  to 
which  children  seen  through  C.O.P.E.  followed  through  on 
recommended  interventions  was  also  investigated. 

Expected  Outcomes 

Short-term  Outcomes 

1 . Design  of  a service  delivery  model  leading  to  earlier  identification 
of  childhood  mental  health  disorders. 

2.  Indication  of  measures  of  child  and  family  social  functioning  using 
National  Longitudinal  Survey  of  Children  and  Youth  (NLSCY)  to 
facilitate  comparison  of  C.O.P.E.  cases  with  national  benchmarks. 

Long-term  Outcomes 

1 . Improved  patient  outcomes  and  reduced  long-term  social  and 
health  morbidities  associated  with  late  identification  of  mental 
health  disorders. 

2.  Information  to  help  educators  and  health  providers  in  developing 
prevention  and  health  promotion  activities. 

3.  Insight  into  linkages  necessary  to  ensure  timely  access  to  mental 
health  services  for  children  and  families  including  information 
about  barriers  to  access. 

4.  Development  of  a school-based  medical  consultation  model  that 
informs  service  delivery  for  multidisciplinary  mental  health  teams 
working  within  the  Calgary  Regional  Health  Authority  (CRHA). 

Methodology  of 
Project  and 
Independent 
Evaluation 

Project  ImDlementation 

■ Experimental  time  series  design  consisting  of  high-risk  study 
group,  and  high-risk  and  low-risk  control  groups. 

■ Data  were  collected  to  assess  the  influence  of  the  intervention  on 
study  group  children  from  the  children  themselves,  their  parent(s), 
and  their  classroom  teachers.  Information  pertaining  to  the  school 
or  the  Collaborative  Learning  Community  (CLC)  was  collected 
from  the  school  principal  or  the  CLC  Specialist  respectively. 

■ Interviewers  administered  surveys  (closed  and  open-ended 
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questions)  to  study  group  children  and  their  parents  at  two  or  three 
intervals  depending  on  when  the  child  entered  the  study. 

Principals  and  CLC  Specialists  completed  self-administered 
surveys  at  the  outset  of  the  study.  Classroom  teachers  completed 
self-administered  surveys  when  their  student  entered  the  study.  (It 
should  be  mentioned  that  attempts  were  made  to  administer  post- 
surveys to  this  group,  but  low  response  rates  prohibited  this 
analysis.) 

■ Data  collection  also  involved  six  case  studies  selected  by  C.O.P.E. 
staff  from  the  study  group.  Cases  reflected  a range  of 
socioeconomic  status  and  parental  satisfaction  with  the 
intervention.  The  purpose  of  the  case  studies  was  to  gain  insight 
into  children’s  problems  and  parents’  perceptions  of  C.O.P.E. 
processes  and  outcomes. 

■ Data  were  also  collected  from  the  seven  pediatricians  and  five 
child  psychiatrists,  C.O.P.E.  Coordinator  and  Family  Liaison. 

■ Data  were  analyzed  using  SPSS,  Nudist  and  manual 
thematization. 

Independent  Evaluation 

■ Process  review  through  review  of  project  documents  (research 
results,  minutes,  products). 

■ Interviews  with  project  team  members  (Project  Director,  Research 
Director,  Research  Associate,  Research  Administrator,  Project 
Coordinator,  Family  Liaison). 

■ Observation  of  intake/screening  process  (referral  process). 

■ Participation  in  focus  group  conducted  with  medical  team 
members. 

■ Participation  at  various  meetings  with  visiting  experts,  Steering 
Committee,  and  project  team  meetings  (face  to  face  and  via 
teleconference). 

■ Verification  of  interpretation  of  project  findings  by  Project  Director 
and  various  team  members. 

Review  of 

Implementation 

Process 

■ Medical/intervention  teams  were  comprised  of  a developmental 
pediatrician,  child  psychiatrist,  Family  Liaison  and  C.O.P.E. 
Coordinator.  The  intervention  team  interacted  with  school-based 
personnel,  the  child  and  parent(s)  one-half  day  per  week.  Medical 
consultations  focused  on  diagnosis  of  children's  mental  health 
problems  and  were  held  in  four  study  group  schools  which  were 
designated  C.O.P.E.  schools. 

■ Various  methods  were  used  to  inform  schools  about  the  program 
(e.g.,  presentations  to  administrators,  print-based  information). 
Two  primary  intake  periods  into  the  C.O.P.E.  program  occurred 
between  January  and  October  1999.  School  Resources  Groups 
(SRGs)  identified  potential  candidates  based  on  a descriptive 
behavioural  profile  provided  by  C.O.P.E.  (Appropriate  consent 
forms  and  ethics  approval  processes  were  followed.) 

■ The  Family  Liaison  made  regular  contact  with  C.O.P.E.  children 
and  their  families  to  ensure  follow-up  on  the  medical  teams' 
recommendations. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Clearly  articulated  research  design. 

■ Comprehensively  organized  project  team  (Project  Coordinator, 
Research  Director,  Family  Liaison  etc.).  Good  communication 
among  team  members. 
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» Family  Liaison  and  Project  Coordinator  familiar  with  school  culture. 

■ Willing  and  dedicated  medical  teams. 

Challenges 

■ Recruitment  of  control  schools. 

■ Teachers  changing  schools  complicated  follow-up  data  collection 
with  teachers.  Cross-over  of  CLC  Specialists  may  have  led  to 
slightly  compromised  intervention/control  school  comparisons. 

■ The  study  resulted  in  some  increased  workload  for  teachers 
(attending  C.O.P.E.  meetings,  interviews,  etc.). 

■ Research  assistants  had  to  resist  offering  medical  advice  to 
families  during  the  interview  process.  This  indicated  the  level  of 
need  of  study  families. 

■ Desire  of  medical  team  members  for  print-based  information  about 
children  in  advance  of  consultation  meetings. 

■ Maintaining  linkages  between  C.O.P.E.  and  relevant  stakeholders 
took  a great  deal  of  the  Project  Director’s  time. 

■ The  transient  nature  of  the  population  made  follow-up  interviews 
difficult  (approximately  30%  of  study  group  children  moved  or 
changed  schools  during  the  study). 

■ Organizational  changes  (i.e.,  impending  teacher  strike  mid  way 
through  the  project).  This  affected  the  willingness  of  teachers  to  be 
involved  in  the  study. 

■ Lack  of  teacher/principal/Learning  Specialist  responses  to  post- 
surveys limited  the  extent  to  which  analysis  could  be  conducted  on 
data  collected  from  these  groups. 

■ Having  parents  understand  the  project  was  about  diagnosis  not 
treatment  per  se. 

■ Differential  billing  by  psychiatrists/pediatricians  (i.e.,  psychiatrists 
could  bill  for  consultation  time  without  the  child  present, 
pediatricians  could  not). 

■ Sustainable  funding  to  complete  analysis  of  data. 

Results 

■ Earlier  diagnosis  (no  more  than  one  month  elapsed  between  the 
time  of  referral  and  diagnosis). 

■ Accurate  and  comprehensive  diagnosis  by  medical  team. 

■ Parents  in  study  group  had  increased  ability  to  identify 
emotional/behavioural  problems  than  did  parents  in  the  control 
group. 

■ Parents  in  the  study  group  perceived  that  the  C.O.P.E.  intervention 
had  a positive  impact  on  their  children  after  four  months. 

■ Study  group  parents  were  more  knowledgeable  about  resources 
available  to  them  and  they  reported  that  they  were  more  likely  to 
access  them  than  were  parents  in  the  control  group. 

■ Medical  teams  felt  they  were  able  to  make  more  comprehensive 
diagnoses  as  a result  of  group  consultation  involving  parents, 
teachers,  Learning  Specialist,  Family  Liaison. 

■ Improved  communication  between  school/parent, 
school/physician,  physician/parent. 

■ Physician  often  played  the  role  of  mediator  between  parents  and 
the  school. 

■ Professional  boundaries  became  blurred  between  medical  team 
members,  project  team  members,  and  school-based  professionals. 

■ The  Family  Liaison  was  respected  by  parents  and  had  success  in 
encouraging  follow-through  by  parents  on  physicians’ 
recommendations. 
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Teachers  had  an  increased  skill  level  to  present  and  identify 
emotional/behavioural  problems  at  consultation  meetings  with  the 
medical  team. 

Physicians  gained  an  increased  understanding  of  the  school 
culture. 

Changed  practice  patterns  were  reported  by  physicians. 

C.O.P.E.  may  be  more  appropriate  for  non-complex  cases. 

Some  C.O.P.E.  students  (those  requiring  urgent  attention)  were 
absorbed  into  existing  professional  practices  of  medical  team 
members. 

Further  funding  support  is  required  to  complete  data  analyses. 

The  Calgary  Board  of  Education,  CRHA  and  Child  and  Family 
Services  Authority  have  committed  to  support  C.O.P.E. 


Learning 


C.O.P.E.  cases  were  more  severe  than  the  national  sample  of 
children  of  comparative  age  (compared  with  NLSCY  data). 

Some  physicians  have  greater  proclivity  to  working  in  teams  and 
school  community  environments. 

C.O.P.E.  appears  to  play  important  triage  function  in  identifying 
children's  emotional,  behavioural  and  learning  problems. 

Skills  of  medical  team  members  appropriate  for  C.O.P.E.  were 
identified  (e.g.,  flexible  thinker,  team  player,  good  listener, 
understands  child  development  and  learning,  interested  in 
preventative  work,  and  committed  to  children). 

C.O.P.E.  can  serve  as  a training  site  for  physicians,  psychiatry 
residents,  teachers,  and  Family  Liaisons. 

Each  Collaborative  Learning  Community  needs  its  own  Family 
Liaison. 

Information  sharing  across  sectors  is  difficult  (i.e.,  systems  are 
cautious  about  sharing  information  with  each  other). 
Fee-for-service  billing  structure  is  a disincentive  for  delivering 
services  according  to  the  C.O.P.E.  model. 

There  was  an  absence  of  the  next  level  of  care  for  children 
requiring  follow-up  care  and/or  urgent  care. 


Access 


Contribution  to  National  Dimensions 


C.O.P.E.  (school-based  mental  health  program)  provided  increased 
access  to  diagnostic  and  consultation  service  by  medical  team 
members  to  children  with  emotional,  behavioural,  learning  problems. 
Access  to  diagnostic  and  consultation  services  was  provided  in  schools 
within  home  communities. 

Family  Liaison  provided  transportation  to  appointments  and  facilitated 
follow-through  on  physicians’  recommendations. 

Follow-up  care  was  often  not  available  for  children  once  diagnoses  had 
been  made. 


Quality 


Increased  comprehensive  diagnosis  was  achieved  through  a 
collaborative  team  consultation  process  involving  school-based  teams 
and  medical  teams  comprised  of  developmental  pediatricians  and  child 
psychiatrists. 

Teachers  had  increased  abilities  to  present/articulate  children’s  mental 
health  problems  which  facilitated  more  appropriate  diagnosis  and  thus 
improved  quality  of  care  for  children  with  emotional,  behavioural,  and/or 
learning  problems. 

Increased  accurate  information  from  parents  allowed  physicians  to 
make  more  informed  decisions  about  diagnoses  of  children’s  mental 
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health  problems.  Earlier  diagnosis  is  expected  to  result  in  improved 
care  for  children. 

■ C.O.P.E.  may  be  more  appropriate  for  less  complex  cases — that  is, 
those  which  do  not  also  require  social  support  services. 

■ C.O.P.E.  serves  a useful  triage  function  to  identify  children's  mental 
health  problems  and  direct  to  appropriate  interventions. 

integration 

■ C.O.P.E.  demonstrated  successful  collaboration  between  two  key 
systems  (health  and  education). 

■ C.O.P.E.  demonstrated  cooperation  and  collaboration  among  school-  i 
based  professionals  and  medical  professionals. 

■ C.O.P.E.  demonstrated  integration  of  medical  services  with  school 
communities. 

■ C.O.P.E.  demonstrated  that  it  can  serve  as  an  ideal  training  ground  for 
both  medical  practitioners  and  school-based  professionals. 

Health  Status 

■ After  a 4-month  intervention,  study  group  parents  perceived  improved 
emotional/behaviour  in  their  children. 

Cost- 

Effectiveness 

Transferability 

■ Baseline  data  on  a school-based  mental  health  intervention  have  been 
collected  and  are  available  for  comparison  with  other  delivery  models. 

■ Comparison  of  C.O.P.E.  data  with  NLSCY  data  is  available. 

■ The  service  delivery  model  is  transferable  to  school  systems 
everywhere. 

■ Data  collection  instruments  are  transferable  to  other  school  sites. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

■ C.O.P.E.  is  a successful  model  of  community-based  service  delivery  to 
identify  and  address  children’s  mental  health  disorders — especially 
important  for  families  living  in  high-risk  communities. 

■ The  intake  referral  and  diagnosis  systems  are  exemplary  models  of 
collaboration  between  health  and  education  systems. 

■ The  model  demonstrated  how  parents  become  better  able  to  identify 
and  articulate  their  children’s  mental  emotional/behavioural  problems. 

■ C.O.P.E.  model  demonstrates  how  communities  can  assume  greater 
responsibility  to  address  children’s  emotional/behavioural  disorders. 

■ Complex  cases  may  not  be  appropriate  for  the  C.O.P.E.  intervention. 
However,  C.O.P.E.  serves  a useful  triage  function  to  identify  and  direct  ■ 
to  appropriate  levels  of  care. 

■ A fee-for-service  funding  structure  does  not  support  collaborative 
initiatives  like  C.O.P.E.  (school-based  mental  health  programs).  Neither 
does  differential  payment  for  psychiatrists  and  pediatricians. 

■ Information  sharing  across  health  and  education  sectors  needs  to  be 
facilitated  to  fully  realize  the  benefits  of  C.O.P.E. 

■ Indications  are  that  the  C.O.P.E.  school-based  mental  health  program 
benefits  children  with  emotional,  behavioural  and  learning  problems 
through  increased  early  identification.  This  indicates  that  health  and 
education  sectors  may  want  to  collaborate  to  provide  diagnostic 
services  through  medical  intervention  in  schools. 
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Implications  for  " 
Research  and 
Evaluation 


Further  research  is  required  to  investigate  the  impact  of  the  C.O.P.E. 
intervention  on  improved  academic  performance  of  C.O.P.E.  children, 
follow-through  to  treatment,  practice  variability  among  providers, 
utilization  patterns  of  emergency  and  acute  care. 

It  is  important  to  continue  drawing  comparison  to  national  samples 
through  NLSCY. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Prevention/Promotion:  Education 


Misericordia  Health-Lifestyle  Improvement  Education  Centre 

(Demonstration  Project) 


Project  Name 

Misericordia  Health-Lifestyle  Improvement  Education  Centre 

(Demonstration  Project) 

Project  Description 

This  demonstration  project  involved  the  creation  of  a Health-Lifestyle 
Improvement  Education  Centre  at  the  Misericordia  Hospital  within 
Capital  Health  Authority.  Physicians  referred  patients  to  the  Adult 
Education  Centre  for  specific  lifestyle  risk  reduction  information  on 
cholesterol.  The  effectiveness  of  this  approach  and  its  impact  on 
patients’  health  were  evaluated. 

Expected  Outcomes 

Goals 

1 . Enhancement  of  the  ability  of  the  primary  care  physician  to  provide 
Lifestyle  Risk  Reduction  to  patients  without  incurring  cost  at  a 
physician  fee-for-service  level. 

2.  Provision  of  a modern  multi-threaded  objective-based  adult 
education  with  objective  outcome-based  measurements  to 
determine  effectiveness. 

3.  Subsequent  application  of  the  model  to  a community-based 
outreach  program  allowing  patients  without  identified  primary  care 
physicians  to  be  targeted  for  Lifestyle  Risk  Reduction  and  linking 
them  to  family  physicians  who  can  provide  continuity  of  care. 

4.  Identification  of  an  effective  reduction  in  the  lifestyle  risk  identified 

■ reduction  of  cholesterol  levels  to  normal; 

■ initiation  and  maintenance  of  an  effective  fitness  program  for 
patients;  and 

■ reduction  of  the  incidence  of  repetitive  strain  injuries  in  the 
workplace. 

5.  Transferability  of  the  model  to  other  regions  and  jurisdictions  in 
Canada. 

Goals  were  later  translated  into  three  expected  outcomes 

1 . A six-month  program  designed  to  improve  diet  and  physical 
activity  levels  (2-month  course,  4-month  follow  up)  offered  to 
individuals  with  elevated  cholesterol  levels  and  identified  as 
moderate  to  high  risk  for  cardiovascular  disease. 

2.  Clients  demonstrate  change  in  terms  of  decreased  serum 
cholesterol,  improved  diet  knowledge,  improved  dietary  habits, 
improved  fitness  knowledge  and  increased  activity. 

3.  Doctors  have  a place  to  send  clients  who  are  at  risk  for  developing 
cardiovascular  disease,  and  this  intervention  is  easy  for  physician 
and  staff  to  use. 

Methodology  of 
Project 

Implementation  and 

Independent 

Evaluation 

Project  Implementation 

A standard  approach  to  curriculum  development  was  employed  to 
develop  materials  and  design  lectures  for  the  education  program. 
Individual  assessments  of  patients  diet  and  fitness  was 
accommodated  through  individual  consultation  with  professional 
providers  and  trainers. 

Independent  Evaluation 

An  experimental  design  (random  selection,  random  assignment  to 
treatment  and  control  groups)  was  chosen  to  assess  the  effectiveness 
of  the  program.  Pre-  and  post-assessments  were  conducted  on  all 
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participants. 

In  addition,  interviews  with  individuals  in  the  study  (both  physicians 
and  clients)  were  conducted.  These  qualitative  data  were  gathered  to 
better  understand  how  participants  experienced  the  program. 

Review  of 

Implementation 

Process 

Participating  physicians  were  asked  to  identify  patients  with  high 
cholesterol  levels,  and  to  refer  these  patients  to  the  Health-Lifestyle 
Improvement  Education  Centre.  Physicians  were  provided  with  an 
“MD  package”,  that  included 

■ Study  Information  Sheet, 

■ Inclusion/Exclusion  Sheet, 

■ Patient  Demographics, 

■ Patient  Questionnaire, 

■ PAR  Q - Fitness  Assessment, 

■ PAR  Med  X,  and 

■ Patient  Pamphlet  - Informed  Consent. 

Participants  were  initially  asked  to  complete  a lifestyle  factor 
questionnaire,  which  was  coded  and  analyzed  by  the  project  team  to 
categorize  individuals  according  to  their  “stage  of  change”  (i.e., 
according  to  Prochaska’s  Transtheoretical  Model  of  Change). 
Individual  educational  strategies  were  assigned  to  each  participant 
based  on  their  stage  of  change  (pre-contemplation,  contemplation, 
action,  or  maintenance). 

At  the  Centre,  participants  attended  a seven-week  lecture  series 
including  dietary  and  healthy  lifestyle  information.  Upon  completion  of 
the  lecture  series,  participants  were  given  two  weeks  within  which  to 
contact  a personal  fitness  appraiser  (a  certified  fitness  consultant). 

The  participant  received  an  initial  meeting  with  the  appraiser,  and  four 
follow-up  meetings  for  individual  support  and  feedback  on  their 
progress. 

A final  evaluation  component  consisted  of  a two-hour  session  between 
the  participant,  the  fitness  appraiser,  and  the  dietitian. 

Cholesterol  levels  were  measured  before  and  after  participation  in  the 
lecture  series  and  personalized  physical  activity  program. 

Facilitators/ 
Challenges  to 
Implementation 

Ethics  Approval 

■ Obtaining  ethics  approval  for  the  study  was  a challenging  and 
time-consuming  process.  The  process  delayed  project 
implementation  by  over  one  month — a significant  delay  given  the 
existing  short  timelines  of  the  overall  HTF  funding. 

Participant  Recruitment 

■ Recruitment  of  participants  (obtaining  physician  referrals  and 
recruiting  interested  clients)  within  the  allocated  timeframes  and 
study  design  was  also  challenging.  A public  marketing  campaign 
resulted  in  hundreds  of  interested  participants.  However,  most  of 
these  did  not  meet  the  established  criteria  for  elevated  cholesterol 
and  risk  factors  for  cardiovascular  disease. 

Time  Constraints 

■ The  short  timeframe  of  the  HTF  funding  was  a barrier  to  fully 
implement  all  aspects  of  the  project  and  for  demonstrating  positive 
behaviour  change.  A great  deal  of  flexibility  was  required  as  well 
as  sensitivity  to  the  theoretical  underpinnings  of  the  project.  This 
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takes  time,  patience,  and  dedicated  resources. 

Bureaucratic  Barriers 

■ Multiple  layers  of  administration  contributed  to  challenges  in  terms 
of  project  planning  and  developing,  particularly  maintaining  budget 
information.  The  complexities  of  being  situated  as  a demonstration 
project  within  a regional  health  authority  structure,  within  broader 
provincial  and  national  health  departments  each  with  several 
levels  of  administration,  were  particularly  challenging  for  project 
organization  and  management. 

Measuring  a Moving  Target 

■ Changing  algorithms  for  measuring  cholesterol/risk  factor 
calculations  for  cardiovascular  disease,  as  well  as  for  measuring 
healthy  aerobic  fitness,  were  challenging  for  establishing 
consistent  criteria  for  measurement  calculations. 

Results 

Outcome  # 1 

■ Even  with  several  obstacles,  such  as  ethics  and  recruitment,  a six- 
month  program  designed  to  improve  diet  and  physical  activity 
levels  (2-month  course,  4-month  follow  up)  was  offered  to 
individuals  with  elevated  cholesterol  levels  and  identified  as 
moderate  to  high  risk  for  cardiovascular  disease. 

Outcome  # 2 

■ Overall,  both  treatment  and  control  group  clients  demonstrated 
decreased  serum  cholesterol  and  increased  frequency  of  physical 
activity.  Forty-three  percent  (43%)  of  the  participants  in  the 
treatment  group  and  35%  in  the  control  group  achieved  their  target 
LDL  cholesterol  (Low  Density  Lipoprotein). 

■ Treatment  group  clients  demonstrated  some  improved  dietary 
habits. 

■ Statistical  results  must  be  interpreted  with  caution,  however,  as 
there  were  very  few  participants  and  other  issues  related  to  study 
design  discussed  above.  (Long-term  outcome  measures  are 
required  to  assess  sustained  changes  over  time.) 

Outcome  # 3 

■ Family  physicians  who  have  referred  to  the  program  reported  that 
while  they  have  not  changed  their  practice  regarding  elevated 
cholesterol  patients,  they  are  supportive  of  the  program  and  would 
refer  patients  in  the  future  if  the  program  were  to  continue. 

Learning 

Several  key  learnings  emerged  from  this  demonstration  project. 
Learnings  have  been  classified  according  to  the  following  categories: 
organizational/structural  barriers,  operational/procedural  learnings, 
measuring  outcomes  and  maintaining  perspective. 

Ooerational/Procedural  Learninas 

Pilot  Test 

■ The  pilot  test  enabled  the  project  team  time  to  make  modifications 
to  course  presentation  and  delivery. 

Collaboration 

■ The  project  team  learned  that  it  was  possible  to  meaningfully 
collaborate  with  various  health  professionals  to  identify  common 
project  goals. 

Human  Resources 

■ The  project  team  learned  that  recruitment  of  personnel  should 

include  input  from  the  regulatory  body  level  to  avoid  minor  ; 
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communication  and  organizational  issues. 

Teaching  Methods/Content 

■ The  project  staff  learned  that  this  approach  worked  well, 
particularly  that  it  is  possible  to  integrate  the  theory  of  planned 
behaviour  and  the  transtheoretical  model  of  change  into  one 
overall  program. 

National  Initiatives  (Cholesterol  and  Exercise) 

■ The  program  combined  national  initiatives  on  both  cholesterol 
(Canadian  Consensus  Guidelines)  and  healthy  physical  activity 
(Canada’s  Physical  Activity  Guide  and  the  Canadian  Physical 
Activity  Fitness  Lifestyle  Appraisal  System). 

Measuring  Outcomes 
Short-Term  Versus  Long-Term  Change 

■ Both  treatment  and  control  groups  demonstrated  short-term 
changes  in  their  physical  activity  and  cholesterol  levels.  However, 
it  is  important  to  distinguish  between  short-term  and  long-term 
outcomes;  follow  up  will  determine  the  extent  to  which  participants 
sustain  their  change  over  time. 

Selection  Bias 

■ There  was  likely  selection  bias  due  to  the  public  marketing 
strategy.  Participants  were  interested  and  requested  to  sign  up  for 
the  program. 

Hawthorne  (Measurement)  Effect 

■ The  effect  of  being  measured  (i.e.,  being  part  of  the  study),  may 
have  led  some  participants  to  positively  change  their  behaviour. 

Positive  Client  Feedback 

■ Qualitative  inquiry  revealed  positive  client  feedback  regarding 
lifestyle  changes,  not  only  for  themselves  but  friends  and  family 
members  as  well. 

Study  Design 

■ Given  the  small  N a different  research  design  would  have  been 
more  appropriate — for  example,  an  interrupted  time-series  design 
(no  control  group)  where  all  eligible  candidates  could  have 
received  the  education  program. 

Maintaining  Perspective 
Multiple  Sources  of  Information 

■ Beyond  quantitative  findings  multiple  sources  of  information 
contribute  to  project  learnings.  For  example,  qualitative  client 
feedback  and  roundtable  discussions  support  that  participation  in 
the  program  was  a valuable  experience  and  that  clients  made 
positive  lifestyle  changes.  Some  of  these  changes  were  dramatic: 
“Lost  40  pounds.”  Others  were  less  dramatic  but  nevertheless 
significant:  “I  see  the  ideas  of  change  [theory]  applying  to  me.” 

Demonstration  Project 

■ It  is  important  to  remember  that  the  program  was  a demonstration 
project  intended  to  demonstrate  a new  way  of  service  delivery. 

The  program  demonstrated  that  this  type  of  service  delivery  has 
the  potential  to  serve  as  a viable  option  for  clients  with  elevated 
cholesterol.  Physicians  interviewed  were  satisfied  with  the 
program,  and  although  they  have  not  changed  the  way  they  treat 
cholesterol  patients  they  reported  that  they  would  refer  patients  to 

| this  program  in  the  future  if  it  were  to  continue.  Furthermore,  the 
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strict  requirements  of  the  program  can  be  avoided  if  the  project 
were  to  run  as  a regular  program  versus  a research  study. 

Contribution  to  National  Dimensions 

Access 

■ Theoretically,  the  Centre  improved  access  to  continuing  and  holistic 
care  for  clients  with  elevated  serum  cholesterol  beyond  treatment 
offered  in  the  conventional  primary  care  office  setting. 

■ There  appeared  to  be  little  incentive  under  the  current  fee-for-service 
system  for  family  physicians  to  spend  extra  time  with  patients  with 
elevated  cholesterol.  This  program  provided  an  alternative. 

■ If  the  program  continues,  there  is  a waiting  list  of  clients  willing  to 
choose  a diet  and  exercise  program  over  medication. 

Quality 

■ Several  themes  emerged  from  the  data  that  indicated  that  patients 
receiving  the  Invest  in  Your  Health  program  through  the  Centre  would 
suggest  the  Program  offers  an  improvement  in  quality  of  care. 

Increased  Awareness 

The  personalized  approach  removed  the  message  out  of  the  realm  of 
general  mass  media,  social  marketing  context  into  the  personal  experience 
of  the  patient. 

Increased  Knowledge 

Participants  reported  increased  knowledge  about  exercise.  Some 
participants  felt  the  “change  model”  was  also  important  for  them  in 
understanding  the  benefits  of  the  course.  They  had  a better  understanding 
of  the  relationship  between  readiness  to  take  action  and  action.  Several 
participants  reported  continuing  their  commitment  to  change.  Examples  of 
weight  loss  of  13,  15,  25,  40,  and  52  pounds  were  reported.  One  of  the 
most  common  themes  surrounding  knowledge  acquisition  is  that  many 
participants  were  better  able  to  understand  cravings — they  became  less 
guilty  about  these  urges  and  were  subsequently  better  able  to  control  them. 

Power  of  SuDDort 

Support  by  the  cohort  group  in  the  course,  family  support,  and  support  from 
friends  were  cited  as  very  important  in  maintaining  motivation  to  continue 
altering  lifestyle  as  an  approach  to  cholesterol  reduction. 

Self-confidence 

A common  theme  of  self-confidence  emerged  in  conversation  with 
participants.  Many  participants  experienced  growth  in  the  belief  that  they 
could  do  what  was  necessary  to  make  the  changes. 

Integration 

■ Integration  occurred  at  the  program  planning  and  delivery  level, 
whereby  several  professional  groups  were  involved  in  the  project, 
particularly  the  Project  Director  (family  physician),  teaching  staff 
(dietician  and  exercise  psychologist),  and  the  Certified  Fitness 
Consultants. 

■ Integration  within  the  broader  political  and  regional  health  authority 
delivery  system  poses  challenges.  Budget  tracking  was  extremely 
difficult  due  to  several  layers  of  administration  and  requirements  of  the 
funding  agreement. 

Health  Status 

■ The  health  status  indicator  in  this  case  was  the  serum  cholesterol  level  1 
of  participants,  which  was  measured  both  before  and  after  the 
intervention.  Unfortunately,  both  the  treatment  and  the  control  group 
decreased  overall  levels — no  statistically  significant  differences  in  effect  | 
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■ could  be  reported.  However,  a number  of  extraneous  variables  require 

examination  before  a definitive  conclusion  can  be  drawn  concerning 

effect,  for  example 

■ sample  size; 

■ Hawthorne  effect  - implication  of  the  “effect  of  being  tested”  on 
subjects; 

■ difference  in  treatment  and  control  group  cholesterol  levels  at  the 
beginning  of  the  study; 

■ short  duration  of  the  study; 

■ advanced  stage  of  contemplation  of  both  control  and  treatment 
groups;  and 

■ referent  power  of  physicians  over  the  short-term. 

Cost- 

Effectiveness 

■ In  this  study,  insight  concerning  cost  and  effect  is  gained,  but  a detailed 
cost-effective  analysis  was  not  conducted.  However,  the  current  data 
would  assist  an  economic  evaluator  in  determining  the  worth  of  this 
education  approach  relative  to  the  approach  that  was  in  place  before 
the  study  began  or  another  approach  all  together. 

■ The  Misericordia  Health-Lifestyle  Improvement  Program  was  included 
in  a sustainability  study  included  as  one  of  the  studies  conducted  within 
the  umbrella  Primary  Health  Care  project.  Detailed  costing  of  the 
project  is  outlined  in  that  study. 

Transferability 

■ One  of  the  main  goals  of  the  project  was  that  it  could  be  easily 
transferred  to  other  areas  across  the  country.  A portable  package  is 
available  (currently  course  materials  are  contained  in  a series  of 
binders).  There  are  plans  for  this  material  to  be  reformatted  to  an 
electronic  medium  (CD-ROM).  Program  materials  include  program 
planning  information,  patient  manual,  instructor  manual,  PowerPoint 
presentation  slides,  and  complete  references/literature  search 
information. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 


Policy 

Implications 

Given  the  response  to  the  media  release  of  the  education  program,  access 
to  programming  such  as  the  education  program  may  be  in  higher  demand 
than  anticipated — an  economic  evaluation  of  various  alternatives  to 
medication  is  required. 

Implications  for 
Research  and 
Evaluation 

Physicians  and  many  other  health  professionals  perceive  research  methods 
as  experimental.  When  dealing  with  primary  health  care  contexts,  “proof’  or 
evidence  of  success  may  have  to  come  by  way  of  quasi-experimental 
designs. 
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Project  Name 

A Program  Evaluation  of  Diabetes  Centres  in  the  Capital  Health 
Region 

(Evaluation  Project) 

Project  Description 

The  project  was  an  evaluation  of  seven  diabetes  centres  in  the  Capital 
Health  Region  according  to  five  guiding  research  questions  (presented 
below).  Characteristics  of  each  Centre’s  population,  accessibility 
issues,  level  of  satisfaction  with  services,  and  knowledge  attainment 
after  attending  an  education  program  were  addressed. 

Expected  Outcomes 

The  study  was  primarily  exploratory  in  nature  and  was  guided  by  the 

following  five  research  questions: 

1 . What  range  of  services  do  diabetes  centres  in  the  Capital  Health 
Region  provide? 

2.  What  population  does  each  diabetes  centre  serve? 

3.  What  problems  do  clients  and  potential  clients  perceive  as  barriers 
to  accessing  outpatient  diabetes  services? 

4.  How  satisfied  are  clients  with  services  they  receive  while  attending 
a diabetes  centre? 

5.  Are  programs  successful  in  teaching  clients  information  about 
diabetes? 

Methodology 

Program  Impact  Theory  was  used  as  the  basis  for  the  project.  The 

research  team  developed  a model  to  examine  the  outcome  of  client 

participation  from  three  perspectives:  clients’,  clients’  partners,  and 

physicians. 

Data  Collection  Methods 

■ Interviews  with  staff  (at  7 diabetes  centres  across  the  Capital 
Health  Region). 

■ Review  of  documents  and  statistical  information  related  to  the 
number  of  clients  served  and  waitlists. 

■ Convenience  sample  of  clients  attending  a diabetes  education 
program,  and  their  partners,  during  a three-month  period.  (Clients 
and  their  partners  were  surveyed  one  month  after  they  attended  a 
program.) 

■ Clients  were  also  administered  a pre-post  knowledge  test  (before 
and  immediately  following  attendance  at  an  education  program). 

B Clients  with  an  appointment  but  who  did  not  attend  an  education 
program  participated  in  a telephone  interview. 

■ Physicians  practicing  in  Northern  Alberta  were  surveyed. 

■ Study  instruments  were  validated  for  content  with  regional 
representation  from  diabetes  experts,  including  physicians,  nurses, 
and  other  health  professional  groups. 

Review  of 

Implementation 

Process 

Phase  One 

■ An  Advisory  Committee  was  struck  to  guide  the  work  of  the 
evaluation  team.  Initial  focus  on  determining  the  best  approach  for 
evaluating  the  Capital  Health  Region  diabetes  centres. 

■ Stakeholder  consultation  revealed  desire  to  evaluate  all  diabetes 
centres  in  the  region.  Proposal  was  revised  and  included  all  centres 
located  at  hospitals  and  health  centres. 

■ Data  collection  tools  were  initiated  and  site  visits  made  to  all 
centres. 

Phase  Two 

■ The  organizational  structure  involved  two  groups  of  individuals- 
representatives  from  the  regional  diabetes  centres  (advisory 

A Program  Evaluation  of  Diabetes  Centres  in  the  Capital  Health 
Region 

(Evaluation  Project) 
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board)  and  the  study  investigative  team  (evaluators). 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Interested  individuals  served  on  Advisory  Committee;  provided 
valuable  direction  and  support 

Challenges 

■ Time  restrictions  limited  the  scope  and  methods  for  the  study. 

Results 

1.  What  range  of  services  do  diabetes  centres  in  the  Capital 

Health  Region  provide? 

■ All  centres  offered  programs  of  varying  lengths  (from  one  to 
four  day  programs);  one  centre  offered  an  evening  program. 
Different  program  options  were  available  for  each  diabetes 
type.  For  example,  programs  were  offered  as  diet,  diet  and 
pills,  pills  only,  pills  and  insulin,  and  insulin  only.  The  evaluator 
also  reported  the  range  of  follow-up  services,  client  costs, 
staffing,  and  locations  across  the  diabetes  centres. 

2.  What  population  does  each  diabetes  centre  serve? 

■ The  proportion  of  clients  who  visited  Capital  Health  centres 
from  other  health  regions  ranged  from  13%  to  81%. 

3.  What  problems  do  clients  and  potential  clients  perceive  as 
barriers  to  accessing  outpatient  diabetes  services? 

GeoaraDhv 

■ Non-Edmonton  residents  indicated  they  would  prefer  closer 
access  to  their  place  of  residence  (e.g.,  a mobile  clinic  for  rural 
patients). 

Time 

■ Clients  reported  wait  times  between  a few  days  to  five  years. 
While  waiting  to  attend  the  education  program,  18%  clients  saw 
another  doctor  about  their  diabetes.  The  most  common  reason 
that  “no-shows”  (clients  with  appointments  to  attend  the 
program  but  who  did  not  show  up  were  interviewed  by  phone) 
did  not  attend  was  work  commitments,  followed  by  having  no 
say  in  appointment  times  (these  clients  would  have  preferred 
evening/weekend  times).  The  top  priority  among  physicians 
surveyed  was  to  decrease  the  waiting  time  for  patients  who 
need  diabetes  education. 

Finance 

■ Reducing  the  cost  or  offering  programs  at  no  cost  was  the 
second  most  frequently  reported  factor  that  would  make 
program  attendance  easier  for  clients.  Besides  program  fees, 
costs  included  parking,  transportation,  and  childcare.  “No- 
show”  clients  reported  costs  associated  with  missing  work  as  a 
barrier.  Physicians  also  reported  that  cost  to  attend,  especially 
for  rural  patients,  was  a barrier  to  access.  Physicians  noted  the 
need  for  additional  resources  and  funding  for  the  centres, 
especially  in  rural  areas. 

4.  How  satisfied  are  clients  with  services  they  receive  while 
attending  a diabetes  centre? 

■ Overall,  clients  were  very  satisfied  with  the  services  they 
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received  six  weeks  after  attending  a diabetes  education 
program,  including  services  from  the  physicians,  nurses, 
dieticians,  and  desk  clerks.  90%  of  clients  and  93%  of  their 
partners  were  satisfied  with  the  clients’  involvement  in  the 
program. 

5.  Are  programs  successful  in  teaching  clients  information  about 
diabetes? 

■ Clients  not  taking  insulin  responded  to  14  questions  before  and 
after  the  program.  Their  median  score  increased  from  10  to  12 
(statistically  significant  based  on  Mann-Whitney  non- 
parametric  test).  Clients  taking  insulin  responded  to  23 
questions  before  and  after  the  program.  Median  scores 
increased  from  17  to  18,  although  this  difference  was  not 
statistically  significant.  Scores  were  not  dependent  on  which 
centre  clients  attended  (for  either  group).  Clients  and  partners 
were  also  asked  to  list  observations  about  their  educational 
experience.  The  number  one  response  for  both  was  related  to 
knowledge  gain. 

Learning 

■ The  evaluator  acknowledged  the  value  of  recruiting  interested 
individuals  to  serve  on  the  Advisory  Committee  (they  provided 
valuable  input). 

■ More  time  needed  for  longer-term  outcome  studies. 

Contribution  to  National  Dimensions 

Access 

■ The  evaluator  noted  several  issues  related  to  accessibility  to  the 
diabetes  education  programs.  These  included  when  the  programs  were 
offered,  the  length  of  time  between  when  a consult  was  made  and  when 
a client  attended  a program,  location  of  the  centres,  and  costs 
associated  with  attending  a program. 

■ The  diabetes  education  programs  across  the  Capital  Health  Region 
varied  with  regard  to  the  type  and  length  of  programs  offered,  follow-up, 
and  program  fees. 

■ The  evaluator  suggested  that  access  could  be  improved  with  more 
flexible  programming  including  evening  and  weekend  programs,  centres 
in  areas  outside  Edmonton,  ongoing  support  for  clients  before  and  after 
they  enter  a program  and  reminder  notices  because  of  the  time  lapse 
between  the  booking  and  appointment  time.  Also,  a promotional 
campaign  that  focuses  on  strategies  for  enhancing  the  profile  of  the 
diabetes  education  services  would  be  beneficial. 

Quality 

Satisfaction 

■ Clients,  their  partners,  and  physicians  were  satisfied  with  the  services 
provided  by  the  Capital  Health  Region  diabetes  centres. 

Course  Content 

■ Clients  not  requiring  insulin  demonstrated  increased  knowledge  about 
diabetes  after  attending  an  education  program.  However,  clients 
requiring  insulin  did  not  show  improvement.  The  evaluator  suggests  that 
the  content  and  delivery  of  educational  programs  for  individuals  requiring 
insulin  needs  to  be  reviewed.  A more  comprehensive  program  might  be 
considered  for  those  with  no  previous  education  and  a shorter  program 
for  those  with  a good  knowledge  base. 

Integration 

■ There  was  a lack  of  awareness  among  physicians  and  consumers  as  to 
the  range  of  diabetes  education  services  with  the  Capital  Health  Region. 

■ Communication  between  referring  physicians  and  diabetes  centres  was 
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■ not  always  optimal.  The  evaluator  recommended  enhancing  and 
standardizing  communication  between  referring  physicians  and  diabetes 
centres. 

■ Peer  support  was  an  important  aspect  of  the  diabetes  education  centre 
experience. 

■ The  evaluator  recommended  vertical  integration  of  the  Capital  Health 
Region  diabetes  centre  services  (e.g.,  clinical  protocol  development, 
medical  records  uniformity,  and  more  coordinated  programming). 

Health  Status 

■ As  a result  of  attending  a diabetes  education  program,  clients  reported 
changes  to  their  health  and  knowledge  about  diabetes.  Specifically, 
clients  not  taking  insulin  showed  an  increase  in  knowledge  (clients 
requiring  insulin  did  not  increase  knowledge  related  to  diabetes). 

■ After  attending  the  program,  the  percent  of  clients  who  indicated  that 
their  blood  sugar  remained  out  of  control  decreased  from  64%  to  22%. 

■ Most  clients  indicated  that  they  exercised  daily,  tested  their  blood  sugars 
regularly,  and  took  insulin  as  directed. 

■ The  determinants  of  health  were  used  as  a component  of  the  framework 
for  the  study.  Social  and  Economic  Environment  - peer  support  was  an 
important  aspect  of  the  education  experience.  Physical  Environment - 
location  of  the  centre  was  an  issue  for  many  respondents  (preferred  a 
centre  closer  to  home).  Personal  Health  - clients  were  making  changes 
such  as  improved  eating  habits,  exercising,  and  monitoring  blood 
sugars.  Individual  Capacity  - clients  felt  that  they  had  a better  sense  of 
control  over  their  diabetes. 

Cost- 

Effectiveness 

Transferability 

■ The  evaluator  indicated  that  the  study  was  exploratory  and  thus 

generalizability  to  other  populations  should  be  done  with  caution.  The 
instruments  were  piloted,  but  reliability  and  validity  studies  were  not 
conducted  for  all  instruments. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Implications 

Regionalization  - Many  centres  provided  services  to  clients  residing  outside 
the  Capital  Health  Region.  The  relationship  and  differences  between 
regionalization  and  integration  must  be  addressed. 

Policy 

Fundina 

■ Program  fees  and  other  costs  (transportation,  parking,  and  childcare) 
were  a barrier  to  access  for  many  clients.  Physicians  also  reported  that 
financial  costs  were  a barrier  for  clients.  Some  respondents  felt  that  the 
diabetes  education  programs  should  be  publicly  funded. 

Research 

Ethics  Review  Process 

■ The  study  underwent  two  ethics  review  processes  (Health  Research 
Ethics  Board  B and  Alberta  Health  and  Wellness).  This  can  be  a time 
consuming  process. 

■ Diabetes  Centre  staff  reported  that  obtaining  client  and  partner  consent 
was  time  consuming. 

Future  Research 

■ The  evaluator  recommends  directions  for  further  inquiry.  These  include 
longitudinal  studies  using  multiple  indicators  of  behaviour  change; 
studies  to  target  interventions  related  to  follow-up  and  support;  study  of 
different  models  of  delivery  (education)  and  comparison;  studies  of  the 
people  who  do  not  attend  programs,  and  research  regarding  cultural, 
language,  and  literacy  barriers  to  attending  a diabetes  centre. 
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Evaluation 

■ The  evaluator  noted  that  the  structure  of  the  evaluation  should  be 
participatory  and  collaborative. 

■ Key  stakeholder  support  was  critical  (e.g.,  contact  persons  responsible 
for  facilitating  data  collection  at  each  of  the  7 centres). 



■ Evaluation  was  based  on  established  program  evaluation  and  behaviour 
change  theories. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Integrated  Service  Delivery 


Project  Name 

Lakeland  Integrated  Community-based  Palliative  Care  Primary 
Health  Care  Project 

(Demonstration/Evaluation  Project) 

Project  Description 

This  demonstration  and  evaluation  project  was  designed  to  improve 
access  to  a broad  range  of  palliative  care  services  in  Lakeland 
Regional  Health  Authority,  improve  the  quality  of  care  to  palliative 
patients,  and  develop  an  integrated  regional  system  for  the 
management  of  palliative  patients.  This  was  a two-phase  project — 
development  and  evaluation.  Stage  one  involved  the  development  of  a 
model  of  integrated  delivery.  Stage  two  involved  piloting  and 
evaluating  the  model  of  delivery  in  the  community  of  Bonnyville. 

An  integrated  service  delivery  model  was  developed  that  included 
multi-disciplinary  team  of  providers  (physicians  and  staff  at  Bonnyville 
Health  Centre),  Home  Care/Acute  Care  nurses,  Rehabilitation, 
Pharmacy,  Respiratory  Services,  social  worker,  volunteer  coordinator 
and  funeral  director.  A Physician  Consultant  and  Nurse  Consultant 
were  key  members  of  the  team.  They  provided  consultation  advice  to 
other  physicians  and  all  nurses.  A central  intake  registry/database 
established,  regional  referral  system,  Policies  and  Procedures  Manual, 
assessment  tools  were  developed,  Palliative  Care/Acute  Care  Plan 
and  Communication  Passport. 

Expected  Outcomes 

1 . Development  and  implementation  of  a regional  case  management 
system  for  the  delivery  of  integrated  palliative  care  services  in 
Lakeland; 

2.  Development  of  a management  and  information  system  for 
ongoing  monitoring  and  evaluation  of  palliative  care  service;  and 

3.  Improved  quality  of  care  for  palliative  clients. 

Methodology  of 

Independent 

Evaluation 

Qualitative  methodology  comprised  of  site  visits,  observation  at 
training  sessions,  document  review,  ongoing  communication  with  the 
Project  Coordinator  (Nurse  Consultant)  and  Project  Director  (Physician 
Consultant),  attendance  at  regular  project  team  meetings  (face  to  face 
or  teleconference),  interviews  with  project  team  members,  and  various 
other  project  participants  including  nurses,  Directors  of  Nursing  (Acute 
Care,  Home  Care),  survey  of  nurses  and  physicians.  Collaborative 
development  of  survey  instruments  with  internal  evaluator  (who  left  the 
project  prior  to  project  close),  and  analysis  of  interview  data  collected 
by  the  Research  Assistant  internal  to  the  project  from  caregivers  of 
palliative  patients. 

Review  of 

Implementation 

Process 

■ Project  was  based  on  a comprehensive  assessment  of  community 
need  with  respect  to  palliative  care. 

■ Baseline  information  was  collected  with  respect  to  availability  of 
respite  beds,  equipment,  service  delivery  patterns,  and  staff 
expertise  in  palliative  care. 

■ An  integrated  service  delivery  model  was  developed  that  included 
multi-disciplinary  team  of  providers  (physicians  and  staff  at 
Bonnyville  Health  Centre),  Home  Care/Acute  Care  nurses,  Rehab, 
Pharmacy,  Respiratory  Services,  social  worker,  volunteer 
coordinator  and  funeral  director.  A Physician  Consultant  and 
Nurse  Consultant  were  key  members  of  the  team.  They  provided 
consultation  advice  to  other  physicians  and  all  nurses  and  made 
themselves  available  to  families  either  in  person  or  via  telephone. 

Lakeland  Integrated  Community-based  Palliative  Care  Primary 
Health  Care  Project 

(Demonstration/Evaluation  Project) 


rA 
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■ A Regional  Palliative  Care  Advisory  Committee  formed  to  support  1 

the  project.  Community  Advocacy  Groups  existed  across  the 
region.  These  were  primarily  advocates  and  patient  supporters  i 

and  provided  equipment  and  furniture  for  palliative  rooms  in  the 
Health  Centre.  !; 

■ Intake  of  palliative  patients  involved  assessment  and  referral  by  a 
physician  (usually  family  physician  but  also  by  cancer  specialists) 
through  a central  intake/referral  system. 

■ Training  sessions  were  offered  at  several  locations  across  the  ! 

region  to  regional  staff  in  pain  and  symptom  management  and  to 
introduce  staff  to  the  Physician  Consultant. 

■ Palliative  Care  Policies  and  Procedures  Manual  (including  vision, 
definition  and  set  of  principles,  regional  standards  reflecting  those 
of  the  Canadian  Palliative  Care  Association  for  palliative  care)  was 
developed  with  input  from  multiple  care  providers. 

■ Standardized  charting  procedure  was  developed  and  piloted. 

■ A Communications  Passport  (outlining  treatment  plans)  was 
developed  that  patients  kept  with  them  to  share  with  their  care 
providers. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Having  the  Medical  Director  part  of  senior  management  helped  to 

build  support  for  the  project  by  updating  and  educating  senior  | 

administrators  and  Board  members  about  palliative  care  and  the 
project. 

■ Support  from  the  region  was  evident  with  palliative  care  being 
identified  in  the  Business  Plan  with  intent  to  fund  the  program. 

■ The  Manager  Health  Services  managed  Home  Care,  Acute  Care 
and  Continuing  Care,  and  as  such,  was  aware  of  the  breadth  of 
resources  available  to  the  pilot.  The  Manager  saw  and  supported 
several  linkages  between  these  sectors,  and  helped  the  project 
team  frame  its  thinking  about  palliative  care  in  relation  to  its 
management  and  administration. 

■ The  Edmonton  team  served  as  mentors  to  the  Lakeland  project. 

■ The  Nurse  Consultant  shared  a similar  knowledge  base  as  the 
Physician  Consultant.  This  facilitated  communication  between 
them  and  with  other  providers.  The  Nurse  Consultant  was  known 
and  respected  by  the  physicians.  This  helped  their  acceptance  of 
her. 

■ The  Nurse  Consultant  and  Physician  Consultant  developed  a 
positive  communication  system  between  them  as  well  as  a positive 
working  relationship. 

Challenges 

■ Organizational  restructuring  meant  formation  of  a new  project 
team  and  selection  of  one  pilot  site  rather  than  region-wide 
implementation  of  the  integrated  service  delivery  model. 

■ There  was  a decrease  in  number  of  Registered  Nurses  at  the  pilot 

site;  only  RNs  could  administer  medication  (most  Home  Care 
nurses  were  LPNs).  j 

■ Turf  protection  (physicians  and  nurses  seeking  advice  from  the 
Physician  Consultant  and  Nurse  Consultant). 

■ Some  scope  of  practice  issues  emerged  in  that  in  rural  practice 

physicians  were  unable  to  bill  for  telephone  consultation 
(according  to  present  standards  in  Alberta).  j 

■ For  the  project  coordinator,  balancing  role  of  consultation  with 
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project  coordination  was  burdensome. 

■ Not  all  physicians  in  the  region  were  aware  of  the  pilot  and  thus 
some  palliative  patients  who  may  have  benefited  from  the  advice 
and  care  of  the  Physician/Nurse  Consultants  were  not  referred  to 
the  pilot. 

Results 

The  following  products/processes  were  developed:  central  intake 

registry/database  established,  regional  referral  system,  Policies  and 

Procedures  Manual,  assessment  tools  were  developed,  Palliative 

Care/Acute  Care  Plan  and  Communication  Passport. 

Signs  of  success 

■ Physicians  alerted  the  Consultation  Team  to  anticipated  referrals. 

■ Requests  for  consultation  from  physicians  outside  the  pilot  and 
also  from  outside  the  region. 

■ Overcoming  barriers  to  turf  issues  between  physician  and 
physician  and  physician  and  nurse. 

■ Rural  physicians’  acknowledgement  that  the  Nurse  Consultant 
was  an  important  part  of  the  team. 

■ Opportunity  to  apply  new  knowledge  and  demonstrate  positive 
impacts  of  the  pilot  on  patients  to  other  staff. 

■ Consideration  by  physicians  and  other  caregivers  (e.g.,  nurses)  of 
alternatives  to  conventional  palliative  care  treatment. 

■ Patients/caregivers  experienced  difficulty  accessing  pain 
medication  from  pharmacists. 

■ Linkages  were  made  to  palliative  care  groups  throughout  the 
region,  with  other  regions,  and  with  the  Alberta  Cancer  Board.  The 
Project  Coordinator  was  invited  to  join  the  Palliative  Care 
Association  of  Alberta;  the  Physician  Consultant  the  Alberta 
Cancer  Board  Advisory  Council.  Several  requests  for  information 
have  originated  across  the  province,  and  one  from  the  US. 

■ Families/caregivers  were  very  satisfied  with  care.  Families  found 
coping  with  the  death  easier  when  the  patient  died  at  home.  They 
were  especially  satisfied  with  the  level  of  care  provided  by 
physicians  and  the  Nurse  Consultant. 

■ Physicians  felt  they  provided  better  quality  care  through  improved 
pain  management  control,  expanded  knowledge  about  palliative 
care  and  improved  access  to  quality  care  for  patients.  Similarly, 
nurses  felt  they  provided  improved  care  largely  as  a result  of 
improved  pain  management  and  coordination  of  care  among 
providers  (improved  communication). 

■ Changed  practice  was  evident  in  greater  interaction  between 
home  care  and  acute  care. 

■ Physicians  had  increased  understanding  of  pain  and  symptom 
management,  narcotic  rotation  and  terminal  sedation  through 
collaborative  decision  making. 

■ The  pilot  demonstrated  increased  involvement  of  other  providers 
(e.g.,  social  worker,  physical/occupational  therapists)  in  the  care  of 
palliative  patients. 

■ The  pilot  also  resulted  in  increased  skill  levels  among  Home  Care 
nurses  in  pain  assessment,  knowledge  about  palliative  care,  pain 
control  and  social  needs  of  families  and  patients  during  end  stage. 

■ Physicians  reported  increased  understanding  of  the  work  and  role 
of  Home  Care  nurses. 
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Learning 

" Bereavement  care  is  an  essential  component  of  holistic  palliative 
care  service.  The  pilot  highlighted  the  lack  of  such  care  in  the 
region. 

■ Support  for  ongoing  consultation  is  key  to  changing  palliative  care 
practice. 

■ A trusting  relationship  between  those  who  seek  consultation  and 
those  who  provide  it  is  essential. 

■ Support  for  the  role  of  Nurse  Consultant  and  a Palliative  Care 
Coordinator  is  required. 

■ It  takes  perseverance  to  change  practice  at  the  regional  and 
community  level  with  respect  to  providing  palliative  care  differently. 

■ The  value  of  champions  at  senior  management  and  administrative 
levels  as  well  as  at  nursing  and  community  level  (through 
Community  Advocacy  Committees)  was  regarded  as  key  by  the 
project  team. 

■ Challenges  and  solutions  to  challenges  were  created  by  care 
providers. 

Contribution  to  National  Dimensions 

Access 

■ The  Lakeland  Palliative  project  demonstrated  improved  access  to 

palliative  care  service.  Timely  access  to  appropriate  services  for 

palliative  patients  and  their  families  was  achieved  through  the  following: 

a)  Identification  of  appropriate  palliative  services  through  a community 
needs  assessment  process.  Input  was  gathered  from  multiple 
stakeholder  groups  including  providers  and  families  of  palliative 
patients; 

b)  Direction  of  palliative  patients  to  appropriate  care  through  a central 
client  registry,  referral,  intake  and  assessment  process; 

c)  Common  Communication  Passport  through  which  providers  and 
patients  monitored  care  plans; 

d)  Consultation  with  a Physician  and  Nurse  Consultant; 

e)  Multi-disciplinary,  coordinated  service  provision  (i.e.,  physician, 
Acute  Care,  Home  Care,  respiratory  therapy,  occupational  therapy, 
social  work,  pharmacy,  community  volunteer,  and  funeral  services). 

Quality 

■ 24-hour  access  to  care  led  to  improved  quality  of  care. 

■ The  consultation  team  provided  quality  care.  As  a result  of  the  j 

information  shared  to  other  care  providers,  nurses  felt  they  were 
providing  better  quality  care. 

■ Improved  pain  management  led  to  better  quality  care. 

■ The  social  worker  was  an  important  element  in  providing  improved 
quality  of  care  to  palliative  patients  by  addressing  various  issues. 

Integration 

■ The  service  delivery  model  implemented  in  the  Lakeland  pilot  involved 
the  cooperation  of  physicians,  nurses  (Acute  Care,  Home  Care),  Home 
Support  Aides,  Social  Workers,  occupational  and  respiratory  therapists, 
community  volunteers,  and  funeral  directors.  Coordination  of  multi- 
disciplinary service  delivery  and  palliative  care  consultation  are  two 
separate  tasks  that  were  combined  in  the  pilot.  Coordination  and 
support  for  integrated  service  delivery  requires  that  each  position  be 
regarded  as  a distinct  task  funded  separately,  particularly  since  the 
number  of  palliative  patients  continues  to  rise  and  long-distance  travel 
in  rural  environments  is  the  norm. 

■ The  pilot  also  demonstrated  successful  strategies  to  achieve 
communication  among  providers,  for  example,  through  the 
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Communication  Passport  which  facilitated  the  smooth  transition  of  the 
patient  from  one  care  provider  to  another  (e.g.,  shift  from  Acute  Care  to 
palliative  care  at  home).  The  Communication  Passport  allowed  the 
patient  to  feel  in  control  of  their  care.  It  also  allowed  the  care  provider  to 
know  what  others  had  suggested  with  respect  to  treatment.  As  well, 
communication  across  the  multi-disciplinary  team  was  achieved  through 
adaptation  of  the  recording  system  typically  used  in  the  Health  Center  in 
Bonnyville  (i.e.,  in  relation  to  pain  and  symptom  management).  The 
project  team  learned  that  communication  protocols  need  to  be 
negotiated  with  nurses,  and  that  burdens  of  documentation  must  be 
kept  minimal  for  nursing  staff. 

■ Training  and  education  provided  to  care  providers  and  community 
volunteers  is  key  to  building  capacity  for  providing  services  in  an 
integrated  way.  It  requires  that  providers  know  each  other’s  scopes  of 
practice.  (This  information  has  been  identified  in  the  Policies  and 
Procedures  Manual.) 

■ One  of  the  primary  challenges  to  successful  integration  is  turf  protection 
by  physicians  and  nurses.  The  approach  used  by  the  Consultation 
Team  was  critical  in  encouraging  physicians  to  draw  on  the  expertise 
and  support  of  the  Physician  and  Nurse  Consultant.  Assuming  the 
stance  of  a “humble  expert”  allowed  the  Consultation  Team  to  break 
down  turf  barriers  (physician  to  physician,  physician  to  nurse). 
Personality  of  the  Consultation  Team  was  a key  factor  in  successful 
integration  of  recommendations  and  advice  by  the  Consultation  Team 
into  physicians’  and  nurses’  practices. 

■ With  respect  to  integration  with  other  palliative  care  programs,  the 
Consultation  Team  is  networked  to  provincial  and  national-level 
associations  dedicated  to  advancing  palliative  care.  These  linkages 
keep  the  Consult  Team  informed  of  best  practices  and  allow  them  to 
share  learning  with  other  team  members,  and  incorporate 
advancements  into  the  regional  palliative  care  program. 

Health  Status 

Cost- 

Effectiveness 

Transferability 

■ The  model  of  care  piloted  in  the  Lakeland  project  is  transferable  to 
other  communities  (rural  and  urban),  other  regions,  provinces,  and 
perhaps  countries.  Strategies  for  case  management,  training  and 
education,  and  best  practice  is  available  from  the  project,  for  example: 

■ Lakeland  Regional  Health  Authority:  Policies  and  Procedures 
Manual  for  Palliative  Care  (outlines  scope  of  practice,  clinical 
practice  protocols); 

■ Volunteer  Manual  (outlines  content  and  application  of  skills  for 
volunteers); 

■ Orientation  Manual  for  Care  Providers; 

■ Service  Delivery  Model; 

■ Case  management  system  (i.e.,  charting  procedures/forms);  and 

■ Contents  of  the  training  sessions. 
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Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

An  innovative  and  successful  model  of  community-based  palliative  care 
service  delivery  is  therefore  available  for  application  in  rural  and  possibly 
other  environments.  The  model  is  characterized  by  multidisciplinary  team 
delivery  of  services  coordinated  through  a Palliative  Care  Coordinator.  The 
model  is  capacity  building  in  that  expertise  and  information  are  shared  by 
the  Consultation  Team  (Palliative  Care  Physician  Consultant  and  Palliative 
care  Nurse  Consultant)  across  provider  groups,  as  well  as  with 
families/caregivers  and  community  volunteers.  The  manner  in  which 
recommendations  and  advice  are  offered  by  the  Consultation  Team  (as 
“humble  experts”)  is  key  to  encouraging  and  sustaining  changed  practice. 
Each  encounter  with  the  Consultation  Team,  whether  by  telephone  or  in 
person,  is  an  opportunity  for  learning  about  best  practice  related  to  palliative 
care.  As  the  numbers  of  palliative  patients  increase  and  best  practice 
information  becomes  available,  these  on-the-job  education  and  training 
opportunities  become  increasingly  important. 

Facilitatinq  Inteqrated  Service  Delivery 

Integration  of  services  in  the  pilot  was  facilitated  through  the  creation  of  a 
Communication  Passport  held  by  patients  for  their  own  and  their  providers’ 
reference,  as  well  as  through  the  development  of  protocols  and  mechanisms 
for  common  reporting  formats  for  pain  and  symptom  management.  To  the 
extent  possible,  patients  and  their  families/caregivers  were  involved  in 
determining  appropriate  care.  Community  volunteers  were  organized  to 
participate  in  providing  care  to  patients.  Home  Care  nurses  assumed  a 
variety  of  responsibilities  including  respite  to  families/caregivers.  These 
strategies  demonstrate  the  need  for  common  information  sharing 
mechanisms,  mobilization  of  community  resources,  and  increased 
community  responsibility  for  the  provision  of  holistic,  quality  care  that 
addresses  the  physical,  mental,  and  spiritual  needs  of  the  patient. 

The  pilot  also  demonstrated  that  conventional  practices — those  involving 
pain  management,  for  instance,  may  not  apply  to  palliative  care.  Treatment 
for  palliative  patients  may  require  challenging  instituted  practices  of  care. 

The  project  also  raised  other  system  issues — supporting  patients’  ability  to 
pay  for  medication,  for  example.  Through  the  ingenuity  of  the  social  worker, 
financial  support  was  provided  to  palliative  patients.  System  support  for  end- 
stage  patients  needs  to  be  examined  further. 

ExDandina  the  Professional  Network  1 

As  might  be  expected  in  primary  health  care  endeavours,  innovative 
participants  are  forging  ahead  independently  to  explore  ideas  and  establish 
professional  networks  to  help  them  realize  goals.  The  Lakeland  Consultation 
Team  has  been  invited  to  participate  in  the  Palliative  Care  Research 
Initiative  spearheaded  by  the  Alberta  Cancer  Board.  The  significance  of 
Lakeland’s  accomplishments  have  already  been  acknowledged  by  other 
professionals  and  peers  involved  and  interested  in  improving  palliative  care 
practice  in  Alberta.  Support  for  disseminating  the  results  of  this  pilot  should 
be  encouraged  at  regional  and  provincial  levels. 

Implications  for 
Research  and 
Evaluation 

The  pilot  emphasized  the  importance  of  a comprehensive  Information 
Management  System  to  facilitate  patient  outcomes  with  utilization  data. 
These  comparisons  should  be  encouraged  in  follow-up  study. 
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Project  Name 

Shared  Mental  Health  Care  in  Primary  Care  Practice 

(Demonstration  Project) 

Project  Description 

The  Shared  Mental  Health  Care  in  Primary  Care  Practice  project  was 
established  within  Family  Medicine  and  Mental  Health  Services  within 
the  Calgary  Regional  Health  Authority  to  develop  better  links  between 
family  physicians  and  mental  health  professionals.  The  project  was 
driven  by  the  need  of  family  physicians  and  their  patients  to  have  more 
immediate  access  to  psychiatric  expertise  and  to  improve 
communication  and  collaboration  between  family  physicians  and  mental 
health  professionals.  The  service  model  used  in  the  pilot  represented  a 
combination  of  consultation-liaison  between  psychiatry  and 
behavioural/mental  health  specialists.  The  family  physician  made  the 
initial  contact  with  the  patient  requiring  mental  health  intervention  and 
served  as  the  primary  caregiver.  Physicians  were  supported  by  a 
multidisciplinary  team  that  included  three  consulting  psychiatrists,  three 
mental  health  nurses,  a psychologist,  and  a mental  health  worker 
responsible  for  follow-up  interviews  and  data  collection.  The  project 
was  not  intended  to  change  the  basic  nature  of  primary  care  practice. 
Patient  care  remained  the  province  of  the  family  practitioner.  The  intent 
was  to  expand  the  capacity  of  the  family  practitioner  to  recognize  and 
treat  mental  health  issues  in  patient  care.  The  project  examined  how 
mental  health  services  might  be  better  delivered  within  the  physician’s 
practice  and  office. 

Expected  Outcomes 

Goals 

1 . Delivery  of  mental  health  services  in  a shared  care  arrangement 
with  family  physicians  in  their  offices. 

2.  Implementation  of  a referral  system  to  ensure  better  access  to 
appropriate  mental  health  programs  for  those  patients  in  need  of 
more  specialized  care. 

3.  Development  of  an  integrated  team  of  skilled  health  care 
practitioners  within  a patient-centered,  systemic/biopsychosocial 
framework. 

Methodology  of 

Independent 

Evaluation 

■ The  final  evaluation  plan  reflected  changes  to  the  initial  research 
proposal  (a  controlled  outcome  evaluation  was  deemed  beyond  the 
scope  of  the  demonstration  project). 

■ The  Steering  Committee  agreed  on  a conventional  participant- 
perspective  model  (primarily  satisfaction  measures  and  self 
reports). 

■ Patient  satisfaction  with  their  physician  was  assessed 
(questionnaire  and  interview)  as  well  as  perceived  mental  and 
physical  well-being  (questionnaire). 

■ Physicians  were  interviewed  regarding  satisfaction  with  the  mental 
health  assistance  provided. 

■ Structured  clinical  interviews  were  conducted  with  physicians, 
mental  health  clinicians,  and  psychiatrists. 

■ A combination  of  standardized  questionnaires  used  in  primary  care 
research  was  adopted  as  well  as  a number  of  structured  questions 
administered  during  a post-treatment  interview  to  reflect  the 
following  processes: 

a)  working  relationship  of  the  mental  health  professionals  and 
physicians, 

b)  patient  perceptions  of  the  team  treatment  approach,  and 

c)  patient  perceptions  of  perceived  mental  and  physical  health 
following  treatment. 

70 


Synopsis  of  Independent  Evaluation  Reports 


■ For  most  patients,  the  mental  and  physical  health  outcome 
measure  was  administered  only  at  post-treatment.  During  the 
course  of  the  study  an  effort  was  made  to  collect  a small  sample  of 
pre-post  treatment  health  outcome  measures. 

Review  of 

implementation 

Process 

■ Mental  Health  and  Psychiatric  Services  of  the  Acute  Care  Division 
of  the  Calgary  Regional  Health  Authority  (CRHA)  developed  a 
conceptual  model  of  a shared  mental  health  care  delivery  system 
(1997). 

■ Position  paper  issued  by  the  College  of  Physicians  and  the 
Canadian  Psychiatric  Association  supported  shared  mental  health 
care  within  primary  care. 

■ Family  Medicine  and  Mental  Health  Services  within  the  CRHA 
initiated  a demonstration  study  of  integrated  health  care  (1998).  A 
coalition  of  health  care  leaders  invited  physicians  to  participate  in  a 
discussion  session  of  an  innovative  shared  care  project.  Twenty- 
four  (24)  physicians  agreed  to  participate.  The  evaluation 
component  and  physician  participation  was  funded  by  the  Health 
Transition  Fund. 

■ The  project  began  with  3 consulting  psychiatrists,  a 1 .0  mental 
health  nurse,  and  a 0.6  psychologist.  Two  mental  health  nurses 
were  added  (0.6  and  0.4)  in  May  1999.  A 0.5  psychologist  was 
responsible  for  conducting  follow-up  interviews  and  data  collection. 
This  individual  resigned  in  January  2000  and  was  replaced  by  a 
mental  health  worker. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Previous  commitment/background  work  (conceptual  model,  position 
paper,  leadership  support) 

Challenges  (from  project  report) 

■ Uncertainties  that  all  parties  felt  as  they  entered  the  project. 

Differing  sets  of  expectations  held  by  the  respective  professional 
groups  as  to  their  roles  and  responsibilities.  The  challenge  for  all 
parties  to  set  about  doing  the  work  with  open  minds  and  tolerance 
for  differences  while  mutual  learning  took  place  and  workable 
relationships  formed. 

■ Union  regulations  created  some  delays  in  hiring  staff.  ! 

■ Initial  uncertainties  about  billing  procedures  for  family  physicians 
accessing  funds  through  the  HTF. 

■ Some  inconsistencies  with  obtaining  patient  consent  across 
physician  practices. 

■ Uncertainty  regarding  future  funding  led  to  some  anxiety  regarding 
non-billable  consultation  time  at  project  completion. 

Results 

■ From  December  1998  to  March  2000  a total  of  947  consultations 
involving  463  patients  were  conducted.  The  majority  (81%)  were 
seen  in  direct  contact  by  a mental  health  clinician  and  a family 
physician  while  the  remainder  were  seen  by  one  of  the  three 
participating  psychiatrists  with  a family  physician. 

■ Of  the  344  patients  referred  for  follow  up,  questionnaire  outcome 
data  were  collected  from  143  patients  (108  females,  35  males).  The 
average  number  of  sessions  was  1.4  (range  1-5). 

■ Several  patients  presented  with  multiple  symptoms  and  problems 
making  it  difficult  to  categorize  the  primary  complaint.  The  majority 
of  presenting  problems  were  distributed  among  depression, 
anxiety,  anger,  and  chronic  pain. 

fA 
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Physician  Enablement 

■ A Physician  Enablement  Questionnaire  was  developed  to  examine 
the  degree  to  which  the  primary  care  project  enabled  or 
empowered  participating  physicians  to  better  understand  the  role  of 
mental  health  issues  in  the  care  of  selected  patients.  The  majority 
of  physicians  interviewed  felt  more  enabled  in  terms  of  identifying 
and  managing  mental  health  issues  in  their  practice.  They  felt 
better  able  to  cope  with  mental  health  issues,  better  able  to 
understand  mental  health  issues,  and  better  able  to  help  patients 
with  mental  health  issues.  However,  some  of  the  physicians  felt  that 
they  still  needed  considerably  more  experience  with  shared  care  to 
be  proficient  at  recognizing  mental  health  issues  in  their  patients 
and  knowing  when  to  refer  to  other  settings. 

Structured  Clinical  Interview  - Physicians.  Mental  Health  Clinicians  and 

Psychiatrists 

Reported  job  changes  as  result  of  participation  in  the  project 

■ Physicians  reported  the  following  changes  to  their  job  as  a result  of 
participating  in  the  project:  greater  effectiveness  and  confidence  in 
dealing  with  mental  health  issues,  fewer  overall  and  more 
appropriate  referrals  to  outside  agencies,  improved  diagnostic 
capabilities,  more  time  spent  counseling  patients,  and  busier  as  a 
result  of  spending  more  time  on  mental  health  issues. 

■ Mental  health  clinicians  had  expected  their  role  to  be  primarily  one 
of  teacher  and  discovered  that  in  practice  there  was  more  of  a 
hands-on  role  related  to  patient  care.  They  had  also  anticipated 
more  time  for  case  discussion.  The  psychiatrists  felt  that  the 
challenges  of  consultation-liaison  psychiatry  in  primary  care  made 
their  jobs  more  interesting  in  terms  of  the  skills  required.  They  also 
commented  on  the  difficulties  of  synchronizing  their  schedules  with 
those  of  the  family  physicians. 

Collaborative  consultation  process 

■ Physicians  reported  that  the  collaborative  consultation  process 
included  joint  interviewing  of  patients  by  physician  and  mental 
health  clinician  and/or  psychiatrist,  case  discussion  (patient 
absent),  independent  assessment  by  mental  health  clinician, 
telephone  consultations,  and  theoretical  discussions  of  clinical 
issues.  Mental  health  clinicians  reported  that  a joint  interview, 
directed  by  the  clinician,  was  the  preferred  method  of  shared  care 
as  it  allowed  physicians  the  most  opportunity  for  observing  and 
acquiring  assessment  and  interviewing  skills.  Psychiatrists  also 
reported  using  a joint  assessment,  followed  by  a discussion  of  case 
management  strategies. 

Impact  of  consultation  on  patient  care 

■ Physicians  reported  that  the  consultation  positively  affected  their 
ability  to  manage  mental  health  issues  in  patient  care,  and  reported 
that  the  information  gained  was  generalizable  to  other  cases. 

Mental  health  clinicians  indicated  that  physicians  shifted  from  being 
solely  “prescribes”  and  “fixers”  to  being  enablers  and  facilitators, 
and  seemed  to  prescribe  psychotropic  medication  more 
appropriately. 

Patient  Interview  Data 

■ Over  60%  of  the  patients  were  able  to  describe  their 

symptom/illness  from  a biopsychosocial  perspective. 
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• Interview  findings  reinforced  positive  patient  satisfaction  results. 
Patients  also  commented  on  the  convenience,  accessibility,  and 
absence  of  mental  health  stigma  with  being  treated  in  the 
physician’s  office. 

■ Close  to  70%  of  the  patients  interviewed  felt  that  they  had  shown 
some  improvement. 

Patient  Outcome  Data 

■ The  Short  Form-36  (SF36)  was  used  a measure  of  perceived 
health  status. 

■ Post-treatment  SF-36  data  from  the  patients  in  the  present  study 
were  compared  with  normative  data  from  two  groups  (a  group  of 
chronically  ill  patients  who  were  relatively  well  and  a group  with 
psychiatric  conditions  such  as  depression).  Following  treatment,  the 
shared  care  group  exhibited  scores  between  the  scores  (in  the 
middle)  of  the  other  two  comparison  groups. 

■ Pre-post  SF-36  scores  were  available  for  a small  group  (N=18)  of 
the  present  study  sample.  The  scores  were  higher  at  post  treatment 
suggesting  they  may  have  moved  toward  improved  health  from  pre- 
treatment levels. 

■ The  SF-36  includes  five  symptom/illness  categories.  The  group  that 

demonstrated  the  highest  level  of  post-treatment  functioning  was 
the  Anxiety/Panic  group,  and  the  poorest  functioning  the  Chronic 
Pain/Fibromyalgia/Migraine  group. 


Learning 


The  project  demonstrated  the  successful  integration  of  physicians, 
mental  health  clinicians,  and  psychiatrists  in  a primary  care  setting. 
Participation  in  the  project  contributed  to  increased  knowledge  and 
confidence  among  family  physicians  regarding  managing  mental 
health  issues  within  their  practice  setting. 


Contribution  to  National  Dimensions 


Access  " A maJ°r  objective  of  the  present  study  was  to  identify  a strategy  to 

increase  patients’  access  to  mental  health  services  in  family  physicians’ 
offices.  Patients  commented  positively  on  the  convenience,  accessibility, 
and  absence  of  mental  health  stigma  with  being  treated  in  the 
physician’s  office. 

Physicians  felt  more  enabled  and  appreciated  having  the  increased 
access  through  a)  direct  referral  to  a mental  health  professional,  b)  joint 
session  with  a mental  health  professional  and  a patient,  c)  having  a 
consultation  with  the  mental  health  professional,  and  d)  rapid  referral  to 
appropriate  community  resources. 


Quality 


The  shared  care  model  was  perceived  by  patients  and  by  the  team  of 
health  professionals  in  very  positive  terms. 

The  guantitative  data  from  the  Patient  Satisfaction  Questionnaire 
supported  the  open-ended  data.  Patients  scored  very  high  in  terms  of 
their  satisfaction  with  the  professional  nature  of  their  health  care  team 
and  also  in  terms  of  their  level  of  feeling  understood.  Amount  of  time 
spent  with  the  physician  seemed  to  be  adequate  although  some  patients 
would  have  liked  more  time  with  the  mental  health  professional. 
Physicians  felt  that  this  project  had  increased  their  skill  and  confidence  in 
dealing  with  mental  health  issues  seen  in  family  medicine.  They  reported 
it  had  improved  their  ability  to  recognize,  diagnose,  and  treat  patients 
with  mental  health  problems. 
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■ The  issue  of  how  best  to  deliver  collaborative  therapy  in  a primary  care 
setting  needs  further  study. 

Integration 

■ The  project  was  successful  in  improving  links  between  primary  care  and 
mental  health  care. 

■ Physicians,  mental  health  clinicians,  and  psychiatrists  worked  together 
collaboratively  in  a primary  care  setting.  The  physician  remained  the  first 
point  of  contact  for  the  client. 

■ The  independent  evaluators  noted  that  it  would  be  advantageous  for 
future  integration  to  formalize  the  mix  of  professionals  through  training 
programs,  including  opportunities  for  both  medical  and  non-medical 
health  professionals  (e.g.,  family  practice  clinic  with  opportunities  for 
residents  in  psychiatry,  nurses,  psychologists,  social  workers,  and  other 
mental  health  students). 

■ More  work  needs  to  be  done  to  improve  integration  with  secondary  care. 

■ Changes  reguired  include  becoming  more  interdisciplinary  (blurring  of 
professional  boundaries)  versus  multidisciplinary  (involving  different 
professionals  who  maintain  their  professional  boundaries). 

Health  Status 

■ Close  to  70%  of  the  patients  interviewed  felt  that  they  had  shown  some 
improvement.  The  majority  of  patients  felt  that  the  intervention  improved 
their  ability  to  cope  with  life,  understand  their  illness,  cope  with  their 
illness,  stay  healthy,  have  increased  confidence  about  health,  and  be 
able  to  help  themselves. 

■ The  evaluators  caution  that  given  the  descriptive  nature  of  the  study  and 
the  lack  of  pre-post  assessment,  firm  conclusions  about  treatment 
effectiveness  cannot  be  made. 

Cost- 

Effectiveness 

■ The  independent  evaluator  indicated  that  the  derivation  of  effective 
utilization  or  cost  effectiveness  was  discussed  on  several  occasions 
throughout  the  course  of  the  project  with  external  experts.  In  the  final 
analysis  it  was  deemed  beyond  the  scope  of  this  demonstration  project. 

Transferability 

■ The  main  criterion  of  transferability  was  stated  as  the  identification  of  “a 
model  of  collaborative  practice  to  build  capacity  of  family  physicians  for 
appropriate  recognition,  treatment,  and  referral  of  patients  presenting 
mental  health  problems.”  There  was  consensus  from  all  those  involved 
that  the  delivery  of  mental  health  services  within  primary  care  was 
achieved.  More  work  needs  to  be  done,  however,  before  the  model  can 
be  exported  to  the  national  level. 

■ The  project  involved  24  self-selected  family  physicians  who  exhibited  an 
interest  in  improving  their  understanding  and  delivery  of  mental  health 
services.  There  are  800  primary  practice  physicians  in  Calgary.  More 
physicians  need  to  become  involved  before  one  can  be  confident  in  the 
generalizability  of  the  present  findings. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

■ There  were  no  significant  relationships  between  the  individual  patient 
satisfaction  items,  the  patient  enablement  items,  and  patient  perceived 
health  status  (SF-36  sub-scale  scores).  These  results  indicate  that  high 
levels  of  patient  satisfaction  are  not  predictive  of  high  levels  of  patient 
perceived  health  status.  This  finding  reinforces  the  complex  (nonlinear) 
relationship  between  access  to  health  services,  satisfaction  with 
services,  and  health  outcomes. 

■ All  the  participants  in  this  project  felt  that  patient  care  needs  to  remain 
with  the  physician. 
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Policy 

Implications 

Defininq  Medically  Necessary  Services 

■ There  was  a definite  need  for  active  therapy  in  the  physicians’  practices. 
Depression  was  among  the  most  common  presenting  problems.  Mental 
health  clinicians  initially  expected  their  role  on  the  demonstration  project 
to  be  primarily  one  of  teacher  and  discovered  that  in  practice  there  was 
more  of  a hands-on  role  related  to  patient  care. 

Determinina  Location  for  Care 

■ Some  patients  with  a significant  psychological  component  may  need  to 
be  treated  outside  the  physician’s  office.  Patients  with  chronic  pain 
and/or  fibromyalgia  did  not  respond  well  to  the  treatments  offered  within 
the  shared  care  framework.  These  patients  likely  need  a comprehensive 
interdisciplinary  intervention  outside  the  shared  care  setting. 

Physician  Billina 

■ The  project  level  report  includes  the  recommendation  that  negotiations 
with  Alberta  Health  and  Wellness  and  the  Alberta  Medical  Associated  be 
maintained  with  the  aim  of  ensuring  that  family  physician  consultation 
time  that  is,  at  present,  not  allowed  as  a billable  procedure,  be  included 
in  the  physician  fee  schedule. 

Research 

■ More  females  were  represented  than  males  although  no  significant 
differences  across  gender  were  found  in  terms  of  satisfaction  or 
outcomes. 

■ The  issue  of  how  best  to  deliver  collaborative  therapy  in  a primary  care 
setting  needs  further  study.  For  example,  large-scale  clinical  trials  have 
advocated  treating  chronic  depression  with  a combination  of  cognitive 
therapy  and  antidepressant  medication.  Yet,  formal  cognitive  therapy  is 
an  expensive  and  time-consuming  process,  often  covering  a period  of 
several  months  of  weekly  sessions.  The  majority  of  patients  seen  in  this 
project  received  a single  one-hour  collaborative  session  from  the  mental 
health  therapist.  Most  therapy  in  primary  care  is  pragmatic  and  does  not 
adhere  to  a “pure”  form  of  delivery  (e.g.,  cognitive,  behavioural, 
psychoanalytic). 

■ Future  research  and  practice  will  be  needed  to  determine  the  best  mix  of 
patients,  physicians,  mental  health  professionals,  and  health  systems 
that  produce  the  best  patient  outcomes. 

Evaluation 

■ The  project  level  reports  includes  the  recommendation  that  results  of  the 
HTF-sponsored  evaluation  should  be  reviewed  with  a view  to  retaining 
those  elements  that  could  be  incorporated  into  ongoing  evaluation  of  the 
service.  It  is  recommended  that  future  evaluation  plans  include 
intervention  outcome  measurement  as  well  as  follow-up  of  externally 
referred  patients  to  determine  disposition. 
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1 Project  Name 

Primary  Health  Care  Collectives 

(Research  Project) 

Project 

Description 

The  purpose  of  the  Collectives  research  project  was  to  develop  and 
implement  innovative  primary  health  care  Collectives,  in  which  six  small 
groups  of  providers  (a  family  physician,  a community  pharmacist  and  a 
home  care  case  manager),  developed  and  agreed  on  their  specific  roles, 
developed  lines  of  communication,  and  worked  together  in  a team-based 
approach,  for  the  purpose  of  improving  medication  appropriateness  (i.e., 
correct  dosage  and  use),  improved  patient/client  compliance  with 
prescribed  medication  regimens,  and  medication  management  in  the 
community. 

Expected 

Outcomes 

1.  Improved  medication  appropriateness  (e.g.,  correct  dose,  least 
expensive  alternative  to  others  of  equal  utility). 

2.  Improved  patient/client  compliance  with  prescribed  medication 
regimens. 

3.  Improved  patient/client  health  status. 

4.  Improved  health  care  provider  satisfaction. 

Methodology  of 
Project 

Implementation 
and  Independent 
Evaluation 

Project  Implementation 

■ Primary  data  sources:  Patient  Questionnaire,  Provider  Questionnaire, 
pharmacists’  documentation  of  medication  reviews  and  results  of  case 
conferences,  transcripts  of  team  building  sessions. 

■ Standardized  questionnaires  were  used.  Data  were  coded  and  entered 
into  SPSS.  Transcripts  were  prepared  of  team-building  sessions  with 
professional  facilitator.  Open-ended  qualitative  questions  were  posed 
to  providers  to  assess  attitudes  and  job  satisfaction. 

■ A nonequivalent  control  group  design  was  selected  for  outcome 
measurement  in  that  it  selected  service  utilization  and  humanistic 
outcome  variables  for  which  there  were  both  provincial  and  national 
comparison  data.  The  design,  involving  other  significant  structure  and 
process  variables  such  as  role  recognition  and  acceptance  and 
indicators  of  improved  coordination  are  characteristic  of  case  study  and 
one-group  pretest-posttest  designs  which  are  appropriate  in  projects 
which  try  to  achieve  changes  in  attitude,  behaviour  and  knowledge  not 
likely  to  occur  without  the  intervention.  The  following  measures  were 
used: 

■ Assessment  of  quality  of  medication  use  (appropriateness, 
compliance,  health  status), 

■ Assessment  of  provider  satisfaction, 

■ Team  building  process  (roles,  responsibilities,  communication 
methods), 

■ Presentation  information, 

■ Provider  characteristics  and  behaviours,  and 

■ Seamless  Solutions  software. 

■ A number  of  evaluation  variables  and  analyses  were  conducted  for 
patients/clients,  providers,  as  well  as  two  types  of  economic  data 
(personnel  and  travel  costs  of  Collective  members,  and  self-reported 
health  utilization  by  patients/clients).  For  patients/clients  demographic 
data  were  collected. 

■ To  assess  medication  compliance,  paired  t-test  comparisons  were 
used  to  assess  changes  from  baseline  to  3-  to  6-month  follow  up. 
Repeated  measures  Analysis  of  Variance  controlled  for  age,  sex,  and 
number  of  medications  to  assess  differences  over  time  by  Collective. 

■ To  assess  Medication  Appropriateness  (MAI  was  completed  by  2 
clinicians  for  all  scheduled  medications  for  each  randomly  selected 
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individual),  paired  t-tests  were  used  to  assess  changes  in  the  average 
ratings  at  baseline  and  end  of  project.  I 

■ To  assess  health  status,  the  Short  Form-12  (SF-12)  health  survey  (both 

physical  and  mental  health)  was  used.  Paired  t-tests  were  used  for 
each  measure  to  assess  changes  from  baseline  to  3-  to  6-month  follow 
up.  Differences  between  Collectives  over  time  used  repeated  measures 
of  ANOVA,  controlling  for  age,  sex,  and  number  of  medications.  jj 

■ For  providers  to  determine  age  differences  across  different  groups,  the 

non-parametric  Kruskal  Wallis  test  was  used.  Chi-square  tests  were  !i 

used  on  the  data  reported  as  categories.  Changes  over  time  at 
baseline,  2,  4,  6 months  measured  providers’  perceptions  of 
collaboration,  decision-making  process,  quality  of  patient  care  in  the 
Collective,  job  satisfaction  analyzed  using  repeated  measures  ANOVA. 

■ Economic  data  collected  included  personnel  and  travel  costs  of 

Collective  members  spent  on  various  activities.  ; 

■ Health  services  utilization  by  clients  (self-reported)  was  tracked. 

Changes  in  self-reported  utilization  were  assessed  from  baseline  to  6 
months  using  chi-square  tests. 

Independent  Evaluation 

The  independent  evaluator  attended  project  team  meetings,  interviewed 
the  project  team,  reviewed  documents  on  an  ongoing  basis,  and  conducted 
reflective  sessions  with  the  project  team.  The  independent  evaluation  was 
an  evaluation  of  a research  project  and  provided  both  process  and 
outcomes  assessment. 

Review  of 

Implementation 

Process 

The  Faculties  of  Pharmacy  and  Pharmaceutical  Sciences  and  Medicine 
comprised  the  core  project  team.  The  primary  approach  to  implementation 
was  to  provide  general  directions  and  expectations  (conference  4 patients 
per  week;  recruit  total  of  50  per  Collective,  gather  data,  improve  medication 
use  and  management),  and  leave  local  determination  of  the  specific 
mission,  goals,  communication  style  and  process  to  each  Collective. 

■ 3 teams  (Design  Intervention,  Research  and  Evaluation,  academic 
evaluation  research)  formed.  Weekly  meetings  (Project  Coordinator, 
Project  Assistant,  Evaluator)  were  held. 

■ Recruitment  letters  were  sent  to  4 Family  Physician  Network  leaders  to 
identify  interested  physicians.  A total  of  199  patients  participated  in  the 
project. 

" The  data  collection  process  with  each  provider  and  each  patient  went 
well,  with  a large  return  rate  and  very  few  missing  data. 

■ The  choice  of  targeting  patients/clients  who  were  generally  at  high  risk 
was  appropriate. 

■ The  data  collection  instruments,  strategies,  and  analyses  were  selected 
from  established  and  trustworthy  sources  and  were  a good  choice  for 
this  design. 

■ Quantitative  data  from  the  provider  surveys  were  validated  against 
transcribed  recordings  of  the  group  process  sessions. 

■ The  project  team  added  more  qualitative  data  to  their  quantitative 
information.  Qualitative  information  served  as  sources  of  Continuous 
Quality  Improvement  information  to  be  used  in  a formative  process  with 
the  Collectives. 

■ The  Investigators’  choice  of  instruments  and  some  aspects  of  data 
collection  showed  methodological  innovation  (e.g.,  demonstrated  the 
use  of  Health  Utilities  Index  in  a primary  care  setting;  assessed  the  use 
of  the  Medication  Appropriateness  Index  in  a community  setting;  tested 
and  adapted  to  a community  setting  software  to  record  medication 
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information  and  follow  up  that  was  originally  developed  in  a Winnipeg 
hospital;  and  extended  the  use  of  an  instrument  to  measure 
collaboration  and  satisfaction  about  care  decisions  that  was  originally 
developed  in  an  intensive  care  setting  to  a community  setting). 

Facilitators/ 
Challenges  to 
Implementation 

Challenges 

■ Software  used  to  gather  medication  data  was  not  user-friendly,  nor  set- 
up to  meet  the  needs  of  the  project. 

■ Inadequate  screening  of  Collectives  for  the  project. 

■ Unclear  role  definitions. 

■ Timelines  very  short. 

■ Tight  resources  in  the  home  care  organization  and  lack  of  available 
personnel. 

■ Collaboration,  accessibility,  and  trust  issues  among  some  of  the 
Collectives. 

■ Lack  of  enrollment  by  Collectives  of  patients/clients  not  known  to  them. 

Results 

■ Improved  patient/client  compliance  with  prescribed  medication 
regimens 

■ The  project  achieved  its  intended  outcome  of  significantly  improved 
patient/client  compliance  with  medication  regimens,  with  no 
difference  across  Collectives. 

■ Improved  medication  appropriateness  (e.g.,  correct  dose,  least 
expensive  alternative  to  others  of  equal  utility) 

■ Medication  appropriateness  showed  no  significant  difference  from 
baseline  to  follow  up,  although  only  about  one  third  of  the  data 
were  available  for  analysis. 

■ Improved  patient/client  health  status 

■ Health  status  did  not  significantly  improve,  but  was  maintained 
from  baseline  to  follow  up,  with  no  differences  across  Collectives. 

■ Improved  health  care  provider  satisfaction 

■ Provider  satisfaction  began  at  relatively  high  levels  at  baseline  and 
did  not  significantly  improve  at  follow  up,  with  no  differences  across 
Collectives. 

■ Provider  perceptions  of  their  experiences  in  the  Collectives 
significantly  improved  in  relation  to  their  roles,  working  in  a team, 
receiving  patient/client  information  and  improved  patient/client 
health  status  information. 

■ Improved  access  to  and  coordination  of  pharmacy  and  home  care 
services  with  family  practice  services  for  people  with  complex  health 
needs  helps  to  significantly  improve  their  medication  compliance 

■ Overall  health  status  for  this  generally  unwell  group  was  maintained 
from  baseline  to  3 and  6 month  follow  up 

■ Self-reported  data  on  health  services  utilization,  suggested  a trend 
(not  statistically  significant),  toward  less  use  of  physician  visits, 
emergency  and  hospital  stays 

■ The  time  Collective  members  were  required  to  devote  to  the  project 
was  less  than  expected:  physicians  spent  1.5  hours/week  instead  of  3, 
pharmacists  spent  6 hrs  instead  of  12,  home  care  spent  2.6  instead  of 
8 hours  per  week. 

Learning 

■ Inclusion  of  a control  group  would  have  provided  a means  of 
comparison  and  therefore  increased  generalizability  of  findings. 

■ The  project  should  focus  on  a target  group  of  chronic,  multiple  medical 
conditions. 

■ Extend  the  study  period  to  minimum  of  2 years. 

■ A larger  sample  size  (N=300)  would  help  to  draw  conclusions  that  are 
generalizable  to  larger  groups. 
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■ Allow  follow  up  for  health  status  and  utilization  outcomes  over  time. 

■ Conduct  economic  analysis  when  randomized  controls  are  in  place. 

■ Use  of  the  MAI  (Medication  Appropriateness  Index)  was  explored  in  a 
community  setting.  Additional  analysis  is  required  to  examine  the 
reliability  of  one  of  the  measures. 

■ Characteristics  of  potential  participants:  dissatisfied  with  current 
system,  want  to  practice  differently,  willing  to  commit  time  and  energy, 
willing  to  openly  communicate,  cooperate,  be  flexible,  share  power  and 
decision  making,  and  norms  of  mutual  respect  and  accountability. 

■ Patient  characteristics  which  seem  to  indicate  the  patient  most 
positively  affected  by  the  intervention:  multiple,  chronic  conditions,  high 
users,  ability  for  the  Collective  to  establish  ongoing  communication  to 
facilitate  monitoring  and  follow  up. 

■ Do  collaborative  team  building  before  clinical  responsibility  (groups 
define  their  own  mission  statement,  group  goals  and  communication 
processes  including  conflict  resolution). 

■ Pay  professionals  market  value  for  collaborative  activity  needed  to 
support  patients. 

■ Use  full-time  home  care  and  pharmacist  resources  dedicated  to 
collaborative  practice. 

■ Team  members  should  get  to  know  patients  at  the  same  time  to  build 
corporate  culture  of  “our”  patients. 

Contribution  to  National  Dimensions 

Access 

■ Patients  have  access  to  one-stop  access  to  a team  of  providers  (family  ; 
physician,  community  pharmacist,  home  care  case  manager)  in 
convenient  locations  or  in  the  home. 

■ Patients  have  access  to  home  care  nurse  services  not  previously  j 

available  because  of  ineligibility. 

■ Access  to  collaborative  opportunities  to  case  conference  by  physicians 

and  pharmacists  by  removing  financial  barriers  to  participation.  j 

■ Approval  by  the  Alberta  College  of  Physicians  and  Surgeons  to  have  any 
member  of  the  Collective  enroll  patients  (not  the  AMA). 

■ Increased  access  to  pharmacists  for  patients  with  complex  medication 
problems. 

■ Increased  patient  (with  complex  health  needs)  access  to  community 
resources  and  to  coordinated  non-drug  services  through  home  care  case 
manager  and  to  home  care  for  those  patients  meeting  eligibility  criteria. 

■ Increased  monitoring  and  follow  up  and  facilitated  access  to  other 
community  resources  for  patients  not  meeting  eligibility  criteria. 

■ Access  to  medication  review  in  homes  or  in  convenient  locations  resulted 
in  enhanced  patient/provider  relationships. 

" Increased  access  to  information  by  patients  for  improved  self-managed 
care  (patients  could  phone  a Collective  pharmacist  directly). 

Quality 

■ Statistically  significant  positive  difference  in  the  satisfaction  of  providers 
in  clear  professional  roles,  feeling  part  of  a team,  feeling  that  working 
with  others  is  helpful,  receiving  enough  patient  information,  improving 
patient  health  status  information.  Providers  also  reported  perceptions  of 
positive  impact  of  collaboration  on  quality  of  patient  care,  improvement  in 
medication  use,  improvement  in  patient  health  status. 

■ Unsatisfied  provider  expectation  with  respect  to  decision-making 
processes,  decisions  made,  and  job  satisfaction. 

■ Approximately  two-thirds  of  patients  reported  satisfaction  with  timely  j 

response  to  health  concerns. 
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■ MAI  showed  no  significant  difference  in  average  rating  of  peer  review  of 
medication  appropriateness. 

■ Assessment  of  medication  appropriateness  through  the  MAI  could  be 
included  in  the  regional  Alberta  Health  and  Wellness  Continuing  Care 
Needs  Determination  Instrument. 

Integration 

■ The  Collectives  “coordination  mechanism”  involved  patients  in  the  design 
of  their  care.  Fifteen  percent  (15%)  of  care  issues  in  the  Collective  were 
identified  by  patients. 

■ Improved  integration  and  coordination  of  services  by  improving  the  flow 
and  exchange  of  information  among  the  professionals  dealing  with 
people  with  complex  health  needs  (inside  and  outside  the  Collective). 
Collectives  pharmacists  acted  as  liaisons/consultants  to  the  patients 
regular  community  pharmacist. 

■ Better  use  of  existing  resources  in  assessment  and  treatment  of  patients 
with  complex  health  needs.  Physicians  diagnosed,  monitoring  and  follow- 
up was  done  increasingly  by  non-physician  members  of  the  Collectives 
(i.e.,  pharmacists  and  home  care  case  manager). 

■ Adaptation  of  computer  software  which  enabled  community  pharmacists 
to  document  and  share  information  on  medication  reviews,  health  issues 
identification  and  monitoring  and  follow  up  on  the  issues  in  the 
community  (further  training  of  pharmacists  was  necessary). 

■ Computerized  database  can  be  used  with  other  collaborative  practice 
teams  and  with  other  parts  of  the  health  system  with  proper  safeguards 
for  patient  confidentiality.  Database  would  also  be  useful  for  program 
monitoring  and  service  planning  within  health  regions. 

■ Development  of  mechanisms  to  integrate  publicly-funded  and  non- 
publicly  funded  services  through  obtaining  existing  medications  from 
community  pharmacists  outside  the  Collectives,  communicating  with 
them  as  needed,  advising  them  of  important  medication  issues  for 
patients  who  had  given  consent  to  share  their  medication  review 
information,  and  procedures  to  transfer  medication  lists  from  acute  care 
to  the  Collectives. 

Health  Status 

■ The  project  assessed  changes  in  the  health  of  the  target  population  in 
several  ways.  Since  there  was  no  control  group  in  the  design  of  this 
innovative  and  exploratory  project,  it  is  not  possible  to  draw  conclusions 
form  this  study  that  would  confirm  that  any  changes  were  the  result  of 
the  intervention  solely  as  the  result  of  activity  of  the  Collectives. 

■ No  statistically  significant  difference  in  overall  functional  ability  and 
health  status  (vision,  hearing,  speech,  ambulation,  dexterity,  emotion, 
cognition,  pain).  Two  standardized  tests  of  health  status  were  used 
(Health  Utilities  Index-HUI,  and  SF-12),  which  is  comprised  to  two 
measures:  one  for  physician  health  status  (PCS)  and  one  for  mental 
health  status  (MCS).  Data  were  collected  at  baseline,  3-month  and  6- 
month  intervals.  A repeated  measures  analysis  of  variance  controlled 
fore  age,  sex  and  number  of  medications  showed  no  significant 
differences  over  time. 

■ Of  the  240  issues  identified  by  the  Collectives,  25%  related  to  decline  in 
health  status.  41%  of  these  health  deterioration  conditions  were 
controlled,  improved,  partially  improved  or  resolved,  26%  no  change,  2% 
worse  (decisions  derived  from  lab  tests,  blood  work  etc.). 

■ Qualitative  data  suggested  improvements  in,  for  example,  improved 
infection  due  to  coordinating  medications. 

Cost- 

Effectiveness 

■ The  project  was  not  a cost-effectiveness  study.  Nevertheless,  there  were 
some  results  in  this  project  that  suggested  that  the  intervention  detailed 
here  should  be  further  refined  with  a controlled  study,  followed  by  a 
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formal  economic  evaluation. 

1 . Health  status  of  unwell  group  was  maintained. 

2.  Self-reported  reduced  use  of  health  services  (physician  visits  and 
ER). 

3.  Improved  compliance  over  the  long-term  is  believed  to  be  cost 
saving. 

4.  Avoidance  of  drug  reactions  and  drug  interactions  and  coordinating 
non-drug  related  services  assists  patients  to  better  manage  their 
own  health. 

■ The  Alberta  College  of  Physicians  and  Surgeons  and  Department  of 
Family  Medicine  in  the  Faculty  of  Medicine  and  Dentistry  agreed  in 
principle  about  enrollment  of  patients  in  collaborative  practice  by 
members  other  than  physicians.  This  has  potential  to  increase  cost- 
effectiveness  of  collaborative  practice  models. 

Transferability 

The  following  critical  success  factors  were  identified  for  implementation  of 

this  research  project  to  other  similar  contexts: 

■ Involve  all  professionals  in  the  team  of  investigators  who  originally 
design  the  project  so  that  concepts,  roles/responsibilities/  perspectives  jj 
are  clear. 

■ Involve  additional  stakeholders  in  the  Steering  Committee  (research  and 
policy  bodies). 

■ Provide  ongoing  facilitation  for  the  intervention  groups  and  individuals. 

■ Maintain  communication  among  team  members  and  between  the  project 
and  the  teams  for  motivation,  early  detection  of  provider  dissatisfaction, 
problems,  and  to  promote  accountability. 

■ Include  a control  group  in  the  project  design. 

■ Allow  for  longer  timelines  for  recruitment  (one  year)  and  follow  up  (two 
years)  when  only  one  physician  per  Collective. 

■ Screen  all  potential  Collectives  participants. 

■ Do  collaborative  team  building  before  taking  on  clinical  responsibility. 

■ Focus  on  high  users  of  services  with  chronic,  multiple  medical 
conditions. 

■ Pay  professionals  at  market  value  for  the  collaborative  activity  needed  by 
patients/clients  with  complex  health  needs  to  support  better  outcomes. 

■ Use  full  time  home  care  and  pharmacist  resources  whose  time  is 
dedicated  to  the  collaborative  practice. 

■ Begin  the  collaborative  practice  together. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

The  project  has  contributed  to  understanding  how  collaborative  care  can  be 
effective  for  people  with  complex  health  needs,  what  is  needed  to  support  it, 
and  has  provided  both  researchers  and  clinicians  with  concrete  examples 
and  products  upon  which  others  can  build. 

1 . Project  highlighted  the  need  to  provide  financial  resources  for 
professionals  (pharmacists,  physicians,  home  care  case  managers)  to 
engage  in  team  care  for  patients  with  complex  health  needs.  A re- 
examination  of  Alternative  Payment  Plans  including  fee-for 
comprehensive  care  time  may  be  required  by  physicians  to  serve  people 
who  are  high  users  of  the  health  system 

2.  A re-examination  of  the  payment  structure  to  include  “cognitive  services” 

for  pharmacists  involved  in  team  care  for  complex,  high  needs  patients 
(i.e.,  conducting  medication  reviews,  identifying  drug-related  and  |i 

medication  compliance  issues  and  assisting  in  monitoring  and  follow  up) 
(AHW,  APhA)  is  also  required. 
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3.  Examination  of  advanced  certification  of  “consultant  pharmacist”  (AHW, 
APhA)  which  would  involve  inclusion  of  collaborative  practice  in  formal 
university  preparation  and  training  is  required. 

4.  Examination  of  possible  funding  for  consultant  pharmacists  through 
regional  Home  Care,  funding  by  groups  of  physicians  to  be  part  of  their 
collaborative  practice,  funding  by  AHW  similar  to  rehab  practitioners. 

5.  Examination  of  the  role  of  home  care  case  managers  since  they  played  a 
key  role  in  addressing  the  needs  of  high-needs  patients  who  were  not 
receiving  regular  Home  Care,  or  who  would  not  meet  the  current 
eligibility  criteria  for  Home  Care  services.  Further,  home  care  case 
managers  were  well  suited  to  assist  in  identifying  and  addressing  issues 
in  medication  management,  monitoring  and  follow  up  including 
knowledge  about  community-based  resources  available  to  the  patient. 

6.  This  project  has  implications  for  expansion  of  current  Home  Care  role  in 
Alberta  to  include  “supportive  home  living”  or  “early  intervention” 
community  nursing  units. 

Implications  for 
Research  and 
Evaluation 

Project  findings  point  to  need  to  examine  Medication  Appropriateness  Index 
(MAI)  (developed  mainly  for  pharmacists)  for  its  appropriateness  in 
community  environments. 

The  Collective  project  created  a template  or  model  for  day-to-day  Collective 
activity  which  has  operationalized  the  Joint  Statement  of  the  Canadian 
Medical  Association  and  Canadian  Pharmaceutical  Association  to  enhance 
quality  of  drug  therapy,  but  it  has  expanded  the  model  to  include 
collaboration  with  other  providers.  Other  self-directed  professions  may  find 
this  template  useful  to  explore  dimensions  of  their  own  collaborative  practice. 
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Project  Name 

Integrating  the  Services  of  the  Nurse  Practitioner  in  the  Inner  City 
(CUPS) 

(Evaluation  Project) 

Project 

Description 

The  CUPS  evaluation  project  described  the  integration  process  of  a Nurse 
Practitioner  (NP)  into  the  collaborative  practice  primary  health  care  model 
developed  at  CUPS.  As  well,  the  project  intended  to  service  the  under  j 

served  inner  city  population  of  downtown  Calgary  in  accordance  with  the  ji 

legislation  and  competencies  set  out  by  the  AARN  for  a registered  nurse  !■ 

providing  extended  health  services  in  the  Province  of  Alberta. 

Expected 

Outcomes 

1 . Match  client  needs  with  appropriate  providers. 

2.  Improve  accessibility  to  care. 

3.  Reduce  waiting  time.  ! 

4.  Influence  patient  satisfaction.  ; 

5.  Influence  perceived  health  status. 

6.  Deliver  a favourable  quality  of  care  for  the  designated  target  conditions. 

Methodology 

■ This  project  was  an  evaluation  project.  The  independent  evaluation 
served  as  a review  and  confirmation  of  the  internal  evaluation  process. 
Health  Canada  Program  Consultants’  Guide  to  Project  Evaluation 
(1996)  model  was  used  to  structure  the  independent  evaluation  plan. 

■ The  independent  evaluator  attended  all  meetings  of  the  project 
Steering  Committee. 

■ A variety  of  quantitative  and  qualitative  data  collection  strategies  were 
used. 

■ The  following  instruments  were  developed  and  used  to  collect  data: 
CUPS  client  satisfaction  survey;  health  status  form;  CUPS  daily 
medical  stats  form;  structured  interview  questions.  All  instruments  were 
developed  at  appropriate  low  literacy  levels  for  CUPS  clients. 

■ Client  satisfaction  surveys  were  used  at  intake,  post  move  (CUPS 
expanded  its  location  to  another  floor  in  the  same  building  during  the 
time  of  the  project),  and  post  appointment. 

■ An  appropriate  consent  form  was  signed  by  CUPS  clientele. 

■ Data  were  coded,  entered  EPI-INFO  database,  cleaned,  and  analyzed 
in  SPSS. 

■ Focus  group  was  conducted  with  CUPS  clients  to  gather  their 
perspectives  on  satisfaction  with  services. 

Review  of 

Implementation 

Process 

■ A Steering  Committee  functioned  at  an  ad  hoc  level,  without  a chair, 
and  made  operational  and  evaluation  decisions. 

■ The  CUPS  clinical  staff  met  and  established  guidelines  for  10  medical 
conditions.  Additional  guidelines  will  continue  to  be  developed  and 
added  to  a CUPS  Health  Clinic  Service  Manual. 

■ The  nurse  practitioner  (NP)  at  CUPS  was  the  first  in  Alberta  to  qualify 
under  the  AARN’s  Extended  Practice  Roster  which  was  opened  in  April 
1999.  To  satisfy  provincial  requirements,  the  NP  is  employed  by  the 
CRHA  (the  only  NP  in  this  RHA)  and  seconded  to  CUPS. 

" Delays  resulted  in  the  NP  commencing  his  duties  in  June  1999. 

Because  of  other  CUPS  administrative/supervisory  duties,  the  NP  only 
works  20  hours/week  in  his  clinical  role. 

■ A draft  orientation  guide  for  CUPS  health  clinic  staff  was  developed.  j 

■ In  January  1999,  a focus  group  was  held  with  CUPS  clients  to  identify 
their  perspectives  on  satisfaction. 

■ Between  June  and  April  2000,  client  satisfaction  surveys  were 
administered  to  332  CUPS  clients. 

■ A dictaphone  was  placed  in  the  Health  Clinic  staff  room  and  staff  was 
encouraged  to  make  anonymous  comments  about  the  integration  of  the 
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Nurse  Practitioner  role  and  “decision  points.” 

■ Ongoing  discussion  occurred  between  Steering  Committee  and  CUPS 
Health  Clinic  staff. 

■ Semi-structured  interviews  were  context  analyzed  and  reviewed  with 
Executive  Director). 

■ Minutes  of  Steering  Committee  meetings  were  recorded. 

■ The  Nurse  Practitioner  also  provided  service  in  Outreach  Program  at 
Calgary  Drop-In  Centre. 

■ Receptionist  at  CUPS  acted  as  triage  (tracked  daily  statistics  form, 
client  demonstrations,  tracks  services  provided  and  service  provider). 
Nurse  and  Nurse  Practitioner  columns  added  to  the  intake  form. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ The  presence  of  a project  Steering  Committee  was  seen  as  vital  to  the 
project. 

■ The  group  work  involved  in  drafting  the  clinical  guidelines  served  as  an 
excellent  team-building  activity  for  the  professional  staff. 

■ The  unanticipated  process  to  gain  Provincial  approval  to  implement  the 
role  of  NP  was  successful  because  of  the  willingness  of  all  to  be 
accommodating. 

■ The  willingness  of  the  CRHA  and  CUPS  administrator  to  work  together 
was  pivotal  to  project  success. 

Challenges 

■ Although  a pre-post  design  following  individual  clients  would  have  been 
useful,  the  nature  of  CUPS  clients  prohibited  this  approach. 

■ Planning  for  the  relocation  of  the  CUPS  Health  Clinic  from  the  1st  to  2nd 
floors  impacted  on  the  time  available  to  steering  committee  members 
and  staff  to  participate  in  project  related  activities. 

■ Challenges  related  to  the  instruments  arose  because  of  the  specific 
client  base  of  CUPS  (e.g.,  low  literacy  level). 

■ In  Alberta,  Registered  Nurses  providing  extended  health  services  can 
be  employed  by  regional  health  authority,  Provincial  Health  Board,  or 
provincial  department  only. 

■ No  process  has  been  established  for  the  NP  to  prescribe  drugs  or  order 
lab  tests. 

■ NP  has  no  professional  colleague. 

Results 

■ Although  the  turn-away  rate  was  reduced  and  additional  staff  members 
were  available,  the  total  numbers  of  clients  seen  over  the  2 years  of  the 
project  remained  relatively  stable. 

■ Staff  had  a common  perception  that  the  integration  of  the  NP  role  into 
the  PHC  team  had  improved  access  to  care  of  CUPS  clientele  in 
significant  ways. 

■ Concerns  related  to  access  centered  on  continuity  of  care  with  a variety 
of  physicians  and  clients  noting  that  they  were  not  able  to  consistently 
see  the  same  physician. 

■ 95%  of  CUPS  clients  were  satisfied  with  the  care  they  received. 

■ Clinical  practice  guidelines  for  10  common  conditions  were  developed 
to  facilitate  the  NP’s  role  and  these,  along  with  case  conferences, 
promoted  quality  care. 

■ The  implementation  of  the  NP  role  did  not  greatly  increase  the 
collaboration  within  CUPS  due  to  limited  clinical  time  available. 

■ Coordination  between  CUPS  and  the  local  health  systems  occurred  at 
several  levels. 

■ Evidence  supported  the  transfer  and  utility  of  NP  extended  practice 
skills  from  isolated  northern  settings  to  urban  settings. 

rA 
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■ Roles  between  nurses  and  the  NP  became  blurred. 

Learning 

■ As  changes  occurred  in  the  project,  the  physician  group  was  not 
completely  clear  about  the  role  of  the  NP. 

■ Many  elements  of  the  nature  of  CUPS  worked  in  making  the  project 
successful. 

■ The  elements  of  the  evaluation  design  that  worked  included  the 
simplicity  of  the  design  and  the  fact  that  the  evaluation  did  not 
dominate  the  provision  of  services  to  the  public. 

■ The  use  of  pictorial  and  low  literacy  instruments  were  key  elements  that 
worked  in  the  project  (brevity  key).  These  were  adapted  from 
Dartmouth  COOP  functional  Health  Assessment  Chart  WONCA. 

■ Conventional  health  status  measures  are  inappropriate  for  homeless 
are  marginalized  people. 

■ CPG  development  encouraged  discussion  about  role  clarification.  In 
this  integrated  and  collaborative  model  of  health  service  delivery 
discussions  such  as  these  serve  as  the  basis  of  a continuous  quality 
improvement  process. 

Contribution  to  National  Dimensions 

Access 

■ Clients  saw  appropriate  providers  (e.g.,  clients  from  8tn  and  8th  Health 
Centre  fill  drug  prescriptions  at  CUPS). 

■ The  introduction  of  the  NP  decreased  the  numbers  of  clients  turned  away 
by  40%  and  increased  outreach  to  other  settings. 

■ The  Nurse  Practitioner  increased  the  levels  of  care  available  at  CUPS  to: 
M.D.,  NP,  RN,  LPN. 

Quality 

■ The  collaborative  practice  model  utilizing  a team  maintained  the  high 
quality  of  services  provided  at  CUPS. 

■ The  joint  development  of  Clinical  Practice  Guidelines  laid  the  foundation 
for  a continuous  quality  improvement  process. 

■ Clients  believed  the  NP  provided  high-quality  care. 

■ Physicians  were  able  to  spend  more  time  with  patients. 

■ Client  perception  was  that  wait  time  had  decreased. 

Integration 

■ The  integration  of  the  NP  and  the  implementation  of  the  collaborative 
practice  model  was  the  foundation  for  all  other  project  successes. 

■ Integration  within  CUPS  and  between  CUPS  and  the  health  system  was 
achieved  through  the  NP  by  involvement  in  the  interagency  groups. 

■ CUPS  staff  meet  with  other  inner  city  providers  to  avoid  duplication  and 
promote  continuity  of  care. 

■ There  was  increased  opportunity  for  case  conferencing  between  NP  and 
physician,  and  between  NP  and  other  providers. 

■ CUPS  joined  with  other  health  care  organizations,  detox  centres,  and 
police  services  to  form  Downtown  Interagency  Group. 

Health  Status 

■ Health  Clinic  staff  believed  health  status  of  clients  had  improved, 

although  no  significant  difference  in  health  status  of  12  pre-post  clients 
was  reported. 

Cost- 

Effectiveness 

■ Use  of  collaborative  practice  model  contributed  indirectly  to  cost- 
effectiveness  service  delivery. 

■ The  NP  could  monitor  chronic/complex  cases  between  physician  visits 
thereby  resulting  in  a cost  saving  to  the  system,  particularly  for  clients 
with  chronic  or  complex  conditions 

Transferability 

■ This  was  a case  study  of  one  and  the  unique  nature  of  CUPS  must  be 
considered.  Nevertheless,  several  key  elements  are  conceivably 
transferable,  including 

■ the  support  of  the  CRHA,  CUPS  Board  and  administration,  and 
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CUPS  physicians  as  vital  to  project  success; 

■ a focus  on  team  building  is  an  essential  component  of  successful 
implementation  of  the  NP  role;  and 

■ the  guidelines  developed  for  10  conditions  (e.g.,  sore  throat,  cough) 
are  transferable  to  other  settings  where  NP  role  is  present. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

■ To  sustain  this  and  other  similar  projects  in  Alberta,  there  is  one 
unresolved  challenge  and  that  is  the  lack  of  provincial  government 
approval  for  the  continuing  role  of  the  NP. 

■ NP  can  facilitate  communication  among  various  providers,  increase 
access,  and  improve  quality  (and  perhaps  cost-effective  service 
delivery). 

■ Demo  projects  that  describe  and  evaluate  NP  in  hospital  and  other 
settings  could  build  upon  this  project. 

■ Currently  specialists  are  not  paid  for  NP  referrals. 

■ Public  Health  Act  needs  to  be  amended  to  allow  NP  to  practice  beyond 
authority  of  RHAs,  provincial  health  boards,  and  provincial  departments. 

■ Tracking  the  health  status  of  transient  and  hard-to-reach  clients  requires 
a different  process  than  used  with  other  clients.  Therefore,  proxy 
measures  for  the  special  group  need  to  be  developed. 
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Project  Name 

Strengthening  Multidisciplinary  Primary  Health  Care  Team 
Coordination  or  Chronic  Disease  Prevention  and  Management 

(Demonstration  Project) 

Project  Description 

This  demonstration  project  brought  community-based  health  care 
practitioners  in  the  Calgary  Regional  Health  Authority  (DRHA)  together 
through  a structured  approach  to  improving  the  prevention  and 
treatment  of  two  prevalent  and  important  chronic  health  conditions: 
diabetes  and  osteoarthritis.  The  project  was  conducted  in  two  phases. 
First,  multidisciplinary  working  groups  were  formed,  trained  and 
empowered  with  Internet-based  decision-support  tools.  During  this 
development  phase,  the  groups  considered  best  evidence  about  best 
practice,  identified  chronic  disease  management  problems,  set 
priorities,  prepared  multidisciplinary  care  plans,  and  considered  input 
from  experts.  Implementation  options  were  presented  and  evaluated, 
care  plan  tools  were  selected  and  Phase  2 teams  were  selected. In  the 
second  phase,  team  care  plans  were  implemented  in  six  primary  health 
practices.  Team  function  was  monitored  and  effects  on  processes  and 
selected  outcomes  were  evaluated.  During  on  overlap  period,  results 
from  Phase  1 were  reviewed  by  an  expert  reference  group  and  their 
recommendations  together  with  feedback  from  the  external  evaluation 
team  were  considered  an  acted  upon. 

Expected  Outcomes 

1 . Formation  of  credible,  community-based  PHCT  Working  Groups 
that  understand  the  basic  principles  of  evidence-based  practice  and 
policy. 

2.  Adaptation  of  existing  best  evidence  and  clinical  practice  guidelines 
in  the  form  of  practical  team  care  plans  about  the  management  of 
diabetes  and  osteoarthritis  outside  the  hospital. 

3.  Development  and  use  of  a prototype  online  “communications 
environment”  that  gives  PHCT  Working  Groups  secure  use  of 
electronic  mail,  discussion  forums,  confidential  patient-centred 
team  messaging  and  access  to  expertise. 

4.  Development  and  use  of  online  resources,  including  information 
about  care  maps,  the  target  health  conditions,  clinical  practice 
guidelines,  and  production  of  online  interactive  versions  of  the  two 
project  team  care  plans,  all  integrated  with  the  online 
communications  environment. 

5.  Increased  use  by  primary  care  practitioners  of  health  information 
systems  that  support  health  care  coordination. 

6.  increased  use  by  primary  care  practitioners  of  best  evidence  about 
the  care  of  persons  with  chronic  illnesses. 

7.  Appropriate  multidisciplinary  primary  health  care  teams  were 
formed  for  the  two  target  conditions,  persisted  for  the  duration  of 
the  project,  and  showed  improved  indicators  of  teamwork  and 
collaboration. 

Methodology 

This  project  used  an  action  evaluation  methodology  that  emphasized 
periodic  assessment  and  feedback  to  guide  the  development  of  new 
health  care  processes.  This  method  was  chosen  specifically  to  learn 
how  to  develop  and  strengthen  practice-based  multidisciplinary  primary 
care  team  coordination  of  chronic  disease  management  in  an  Alberta 
fee-for-service  practice  context. 

The  evaluation  methods  used  during  Phase  1 included  the  use  of 
diaries,  focus  groups,  interviews,  and  online  pre-post  surveys,  ranking 
exercises  and  feedback  tools.  In  Phase  2 involved  structured  team 

Strengthening  Multidisciplinary  Primary  Health  Care  Team 
Coordination  or  Chronic  Disease  Prevention  and  Management 

(Demonstration  Project) 
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discussion  and  recording  of  team  member  activity,  diaries,  and  pre-post 
surveys  measuring  different  aspects  of  team  development  and  function. 
The  evaluation  results  of  Phase  1 were  used  to  inform  the 
implementation  of  Phase  2.  During  both  Phases,  qualitative  and 
quantitative  information  about  the  project  processes  generated  by  the 
Working  Groups  and  PHC  Teams  were  shared  amount  participants  and 
helped  to  support  achievement  of  the  project  outcomes. 

Review  of 

Implementation 

Process 

Phase  1 

■ Established  two  multidisciplinary  Working  Groups  to  develop  Team 
Plans. 

■ Developed  Team  Plans  for  multidisciplinary  primary  care  of 
diabetes  and  osteoarthritis. 

■ Created  of  a virtual  team  community  for  each  Working  Group  using 
Internet-based  information  technology. 

Phase  2 

■ Established  six  multidisciplinary  PHC  Teams  to  implement  Team 
Plans. 

■ Team  implemented  Teams  Plans. 

■ Created  a virtual  team  community  for  each  PHC  Team  using 
website-based  information  technology. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

Phase  1 

■ Established  Working  Groups  whose  members  showed  flexibility 
and  adaptability. 

■ Working  Groups  developed  Team  Plans  for  multidisciplinary 
primary  care  of  diabetes  and  osteoarthritis. 

■ Creation  of  a virtual  team  community  for  each  Working  Group  using 
Internet-based  information  technology  to  link  all  members  via  the 
Internet  to  a project  website. 

Phase  2 

■ Successful  establishment  of  six  multidisciplinary  PHC  teams. 

■ Implementation  of  the  Team  plans  and  use,  by  all  Teams,  of  the 
patient  assessment  questionnaire  as  part  of  the  patient  enrollment 
process. 

Challenges 

Phase  1 

■ Short  timeline  for  the  development  of  the  Team  Plans  by  the 
Working  Groups. 

■ Lack  of  consistent  attendance  by  some  Working  Group  members  at 
the  six  half-day  Team  Plan  Development  workshops  made  data 
collection  difficult  with  pre-post  methods. 

■ A significant  time  commitment  for  participating  family  physicians  or 
their  patients  with  no  guaranteed  benefit. 

■ The  isolation  of  primary  and  secondary/tertiary  care  Working  Group 
members  from  one  another  made  it  difficult  to  develop  a 
collaborative  environment. 

Phase  2 

■ Delayed  implementation  of  the  Team  Plans  by  three  months 
because  of  overlap  (33%  to  50%)  between  the  Working  Group  and 
PHC  Team  membership;  additional  time  was  required  for  PHC 
Team  training  and  team  building. 
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■ Time  restraints  for  Project  activities  (i.e.,  Team  meetings  and 
communication  with  other  Team  members,  use  of  website  for 
information  sharing,  follow  up  on  identified  patient  needs). 

■ Ability  of  PHC  Team  members  to  access  their  computers  on  a 
timely  basis  because  while  members  worked  in  the  community, 
their  computers  were  located  in  their  offices. 

■ The  geographic  distribution  of  primary  health  care  providers  in 
CRHA  did  not  favour  multidisciplinary  working. 

■ Patients  did  not  always  qualify  for  primary  health  care  services  they 

were  assessed  as  needing  by  the  Teams  using  the  patient  ' 

assessment  questionnaires. 

Results 

■ The  project  highlighted  the  transition  from  evidence-based  practice 
to  practice-based  evidence.  Recommended  best  practice  was 
transformed  into  tasks  that  could  be  implemented  in  practice  (i.e., 
by  physicians,  nutritionists,  etc.).  In  this  manner  health  “policy”  was 
tested  through  practice. 

■ Working  Group  members  were  able  to  recognize  high-quality 
distillates  and  guidelines  summarizing  best  evidence  and  were 
comfortable  accepting  these  documents  as  a starting  point. 

Working  Groups  proved  particularly  astute  at  determining  which 
guideline-based  recommendations  could  be  adapted  to 
multidisciplinary  team  work  in  the  community. 

■ Team  Care  Plans  were  developed  for  diabetes  and  osteoarthritis. 

■ Working  Group  members  demonstrated  rapid  uptake  of  entirely 
new  information  tools  and  processes  on  the  Internet.  Lack  of  any 
specific  installation  requirements  facilitated  use  of  the  system  in  a 
wide  variety  of  work  settings.  Online  information  tools  were  most 
useful  in  exposing  the  roots  of  disagreement  between  team 
members,  thus  helping  them  to  find  a way  forward  to  collaboration. 
This  value  was  most  apparent  when  the  tools  were  used  during 
fact-to-face  group  sessions.  Members  preferred  face-to-face 
interaction  when  doing  the  work  of  consensus  building  and  care 
plan  review.  Online  activities  could  complement  but  not  replace  the 
workshop-based  activities.  Workshops  conducted  in  a computer 
laboratory  led  to  higher  satisfaction  ratings  with  online  information 
tools. 

■ Although  the  purpose  of  the  project  was  to  strengthen 
multidisciplinary  teams  through  development  and  implementation  of 
team  care  plans,  the  Working  Groups  of  Phase  1 included  more 
supervisory  than  front-line  practitioners.  As  a result,  many 
newcomers  appeared  during  Phase  2 of  the  project,  which 
compromised  some  of  the  work  of  Phase  1 in  team  building  and 
communications  development. 

■ Team  members  were  limited  in  the  time  they  could  give  to  the 
project,  possibly  for  reimbursement  or  other  workload-related 
reasons.  This  prevented  full  engagement  in  any  evidence-analysis 
process  and  limited  the  scope  of  the  team  care  plans  that  could  be 
developed.  Alternative  practice  models  would  be  required  to 
encourage  more  complete  selection,  analysis,  and  adaptation  of 
best  evidence. 

Learning 

■ The  transformation  of  clinical  practice  guidelines  (CPGs)  requires 
translation  into  tasks.  This  is  a separate  an  integral  function  for  the 
successful  application  of  guidelines  to  practice. 

■ Transformation  of  best  evidence  into  best  practice  guidelines 
requires  the  active  participation  of  front-line  practitioners.  Often 
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these  are  the  busiest  health  professionals — most  difficult  to  involve, 
but  essential  because  they  are  the  key  to  successful 
implementation  of  the  guidelines. 

■ On-line  decision  support  tools  provide  a successful  medium  to 
identify  differences  in  perspective  about  best  practice.  The 
technology  allows  for  anonymous  disclosure  of  disagreement. 

■ Health  professionals  require  ongoing  support  to  integrate 
technology  into  their  practices.  This  requires  an  attitudinal  shift 
about  the  place  of  information  in  practice  and  its  influence  on 
practice. 

■ In  practices  that  had  an  existing  information  strategy, 
implementation  of  the  guidelines  (i.e.,  on-line  access  to  the 
guideline,  application  to  specific  cases  through  quick  retrieval  of 
information)  was  most  successful.  These  sites  can  serve  as 
exemplars. 

■ Team  interaction  facilitates  health  professionals  learning  about 
each  other’s  roles.  This  learning  had  to  reoccur  with  the  arrival  of 
each  new  member  on  the  team.  Therefore,  continuity  in  team 
members,  and  commitment  to  participate  is  critical  to  successful 
multidisciplinary  team  development  of  practice  guidelines. 

■ To  demonstrate  regional  support  for  multidisciplinary  involvement 
(and  ultimately,  integrated  service  delivery)  health  professionals 
must  be  reimbursed  for  their  time  to  participate  in  team  activities, 
and  released  from  regular  duties  so  that  participation  is  not  viewed 
as  overload  assignment.  These  incentives  demonstrate  equivalent 
commitment  to  and  support  for  implementation  of  practice 
guidelines  as  for  their  development.  This  kind  of  support  applies  to 
fee-for-service  or  capitated  reimbursement  models. 


Contribution  to  National  Dimensions 

Access 

Quality 

■ This  project  demonstrated  what  attitudes  and  practices  support  the  use 
of  information  to  improve  the  quality  of  health  service.  It  identified  the 
change  management  process  to  support  the  transformation  of  best 
evidence  into  practice-based  evidence,  and  subsequent  application  of 
that  evidence  into  professional  practice  by  multi-disciplinary  teams  of 
providers. 

Integration 

■ There  must  be  a common  understanding  of  the  role  of  each  participant 
on  a multidisciplinary  team  to  build  consensus  around  what  each 
member  of  the  team  can  do  to  implement  a model  of  care  based  on 
practice-based  evidence.  This  information  is  key  to  successful 
functioning  of  multi-disciplinary  teams. 

■ The  project  demonstrated  that  electronic  decision  support  tools  and  on- 
line access  to  information  can  support  multi-disciplinary  involvement  and 
implementation  of  practice-based  care  plans. 

Health  Status 

Cost- 

Effectiveness 

Transferability 

The  participants  identified  the  following  as  being  essential  to  the 
development  of  the  team  care  plans  based  on  practice-based  evidence: 

■ team  members  with  diverse  perspectives  and  from  a variety  of  settings 
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who  focus  on  a common  goal; 

■ assistance  with  learning  the  communication  environment; 

■ facilitation  and  consultation  expertise  in  team  plan  development;  and 

■ an  electronic  communication  environment. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

This  project  has  implications  for  continuity  of  care  for  two  important  chronic 
diseases — diabetes  and  osteoarthritis.  Presently,  poor  continuity  of  care  for 
these  two  conditions  emphasizes  the  need  for  implementation  of  practice- 
based  evidence,  that  is,  best  practice  information  that  practitioners  will  use. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Quality  Improvement 


1 Project  Name 

Evaluation  of  Existing  Primary  Mental  Health  Care  Service  at  the 
Edmonton  Centre  for  Survivors  of  Torture  and  Trauma  (ECSTT) 

(Evaluation  Project) 

Project  Description 

This  evaluation  project  focused  on  a retrospective  evaluation  of  the 
Community  Strengthening  Program,  specifically  the  Early  Intervention 
Program  which  focused  on  non-therapeutic  aspects  of  the  program 
serving  children  in  three  cultural  communities:  Somalian,  Cambodian, 
and  Bosnian.  As  well,  the  evaluation  focused  on  assessing  client 
satisfaction  and  success  of  the  Somali  Women’s  Program.  Evaluation 
also  included  an  assessment  of  the  clinical  program.  The  evaluation 
process  involved  ECSTT  staff  and  volunteers,  Somali  Women,  and 
children  who  had  previously  attended  the  programs. 

Expected  Outcomes 

The  retrospective  internal  evaluation  sought  to  determine  the  success 
of  and  client  satisfaction  with  the  Somali  Women’s  Program,  Early 
Intervention  Program  for  children,  and  the  Clinical  Program.  The  short- 
term goals  of  programs  and  services  offered  at  the  ECSTT  focused  on 
improving  physical  and  mental  well  being.  Long-term  goals  focused  on 
increased  capacity  to  participate  fully  in  Canadian  life  including  social 
and  recreational  activities,  improved  attendance  at  school,  enhanced 
academic  performance,  enhanced  ability  to  secure  and  maintain 
employment,  and  enhanced  ability  to  access  and  appropriately  utilize 
community  resources. 

Methodology  of 
Internal  and 
Independent 
Evaluation 

Internal  Evaluation 

Interviews  with  Somali  women,  program  staff,  volunteers;  focus  group 
discussions  with  children  (Cambodian,  Vietnamese,  Bosnian);  case 
studies  (clinical  program);  document  review. 

Independent  Evaluation 

Document  review,  interviews  with  Project  Director,  Research 
Coordinator,  attendance  at  focus  group  with  teachers,  facilitation  of 
meeting  with  program  staff  to  develop  indicators,  interview  with  a 
Somali  women,  assistance  with  instrument  preparation  and  data 
analysis. 

Review  of 

Implementation 

Process 

■ Key  evaluation  questions  were  prepared  through  an  iterative 
internal  process  involving  Research  Coordinator,  program 
providers  and  volunteers.  Another  iterative  process  was  used  to 
identify  appropriate  indicators/measures  of  success.  This  involved 
program  providers  and  some  school-based  personnel 
(administrators,  teachers,  counsellors)  at  two  schools.  Indicators 
were  used  to  assess  the  influence  of  the  Early  Intervention 
Program  on  children  within  the  school.  The  Independent  Evaluator 
assisted  in  this  process  to  create  of  sets  of  indicators  for  both  the 
Early  Intervention  and  Somali  Women's  program. 

■ Indicators  formed  the  basis  for  the  development  of  interview 
questions  for  children  and  Somali  women. 

■ The  counselor  in  the  Clinical  Program  provided  test  results  and 
case  studies  to  illustrate  the  positive  effects  of  the  counseling 
intervention. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Very  willing  and  eager  school  staff  to  participate  in  the  evaluation 
process. 
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■ Very  knowledgeable  staff  and  volunteers  working  in  Early 
Intervention  and  Somali  Women’s  Programs  (good  practice-based 
knowledge). 

■ Research  Coordinator  very  willing  to  learn  about  evaluation. 

■ Community  Worker  (Somali)  was  instrumental  in  assisting  with 
data  collection  from  Somali  women. 

Challenges 

■ Complete  change  in  project  team  (new  Project  Manager  and 
Research  Coordinator — both  new  to  ECSTT).  Much  project 
information  was  lost  with  the  former  Project  Manager/Coordinator 
who  was  to  serve  as  the  primary  internal  evaluator.  New  Research 
Coordinator  was  new  to  evaluation  practice. 

■ Insufficient  literature  exists  on  indicators  appropriate  for  children  in 
multi-cultural  settings,  particularly  associated  with  refugees  and/or 
children  of  victims  of  torture/trauma. 

■ Children’s  perspectives  of  the  Early  Intervention  Program  relied  on 
their  memories  of  experiencing  the  program,  in  some  cases,  as 
many  as  four  years  prior  to  the  evaluation. 

■ Creating  a balance  between  leading  and  facilitating  the  internal 
evaluation  process. 

Results 

■ List  of  indicators  were  developed  for  the  Somali  Women’s  Support 
Program  and  the  Early  Intervention  Program  (collaboratively 
developed  by  program  staff). 

■ A self-assessment  of  the  Clinical  Program  was  conducted  by  the 
lead  counselor. 

■ Social  support  aspects  of  the  programs  were  most  beneficial  to 
both  Somali  women  and  children.  Important  aspects  of  the 
children’s  program  from  the  children's  perspective  included 
opportunities  for  artistic  expression,  learning  computer  skills, 
discussing  feelings,  playing  with  friends  of  similar  ethnic  origin, 
and  learning  about  Canadian  culture. 

■ The  Community  Worker  was  instrumental  in  supporting  Somali 
women.  However,  there  appeared  to  be  a growing  dependence  on 
her,  which  may  have  inhibited  settlement  of  the  Somali  women 
and  their  successful  integration  into  Canadian  society. 

■ Case  study  results  of  the  Clinical  Program  indicate  that  clients' 
mental  health  status  improved  as  a result  of  clinical  counselling. 

Learning 

■ The  evaluation  process  created  an  opportunity  for  the  ECSTT  to 
build  awareness  of  its  programs  among  schools  and  other  service 
providers. 

■ The  Community  Worker  involved  as  interpreter  in  the  evaluation 
process  may  have  added  a confounding  factor  in  the  validity  of  the 
data  gathered  from  Somali  women. 

Contribution  to  National  Dimensions 

Access 

ECSTT  offered  increased  access  to  mental  health  services  for 
refugees,  immigrants,  and  other  multicultural  groups  through  on-site 
counselors/therapists. 

Quality 

■ 

Clients  (Somali  women)  and  children  liked  the  social  support  offered  to 
them  through  the  Somali  Women’s  Support  Program  and  the  Early 
Intervention  Program,  specifically. 

Outcomes  of  the  Clinical  Program  suggested  that  outcomes  were  being 
achieved,  therefore  clients  experienced  an  improved  guality  of  life  (e.g., 
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sleeping  better,  reduced  anxiety). 

Integration 

■ Somali  women  and  children  attended  classes  at  the  same  time. 

Therefore,  programs  within  ECSTT  appear  to  be  integrated.  However, 
there  is  little  integration  of  ECSTT  programs  with  those  outside  the 
ECSTT. 

Health  Status 

■ Results  of  the  internal  evaluation  of  the  clinical  program  indicated  that 
clients  experienced  improved  physical  and  mental  well  being. 

Cost- 

Effectiveness 

Transferability 

■ Findings  suggested  the  possible  transfer  of  the  Harvard  adaptation  of 
the  clinical  questionnaire  for  survivors  of  torture  and  trauma. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

■ To  encourage  the  integration  of  immigrants  into  Canadian  society, 
opportunities  to  interact  with  groups  of  similar  ethnicity  seem  to  be  an 
important  part  of  creating  a positive  social  support  system  that  can 
create  initial  links  into  mainstream  society  (e.g.,  to  dentists,  physicians, 
etc.).  This  may  be  less  important  for  children  than  adults. 

■ Formative  evaluations  of  programs/services  provided  to  immigrants  and 
refugees  are  necessary  to  ensure  that  programs  are  meeting  clients’ 
needs.  Further,  programs/services  should  ensure  linkages  to  other 
service  providers. 

■ Serious  consideration  should  be  given  to  funding  evaluations  of 
programs  that  are  not  currently  being  offered.  The  level  of  expertise  of 
participants  conducting  internal  evaluations  should  also  be  a criterion  of 
funding  support. 
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Project  Name 

Evaluation  of  Urban  Patients’  Choice  of  an  Emergency  Department 
as  their  first  Contact  with  Primary  Health  Care  Services 

(Evaluation  Project) 

Project  Description 

This  research/evaluation  project  was  conducted  at  the  Peter  Lougheed 
Centre  (PLC)  within  the  Calgary  Regional  Health  Authority.  It 
investigated  why  people  used  the  emergency  department  for  primary 
health  care  needs  and  what  knowledge  users  had  of  alternative 
services.  The  demographic  composition  of  the  multicultural  catchment 
population,  profile  of  patients  using  the  Emergency  Department  (ED)  for 
primary  care,  access  patterns  and  barriers  to  access  were  described 
and  assessed. 

Expected  Outcomes 

1 . Development  of  a transferable  tool  for  use  in  multicultural 
environments  to  measure  patient  perceptions  about  accessibility  of 
primary  care  including  information,  identification  of  barriers  to  use 
of  alternative  primary  care  environments. 

2.  Development  of  a profile  of  patients  using  EDs  for  primary  care. 

3.  Collation  of  patients’  perceptions  about  primary  care  advice, 
access,  and  information  available  at  the  ED. 

4.  Identification  of  barriers  to  the  use  of  alternative  primary  care 
environments. 

5.  Transferability  of  results  to  similar  urban  multicultural  settings. 

Methodology  of  the 

Independent 

Evaluation 

The  independent  evaluation  of  this  research  project  documented 
project  progress  and  outcomes  through  collaboration  with  the  project 
team,  attendance  at  team  meetings,  document  review,  and  ongoing 
critical  reflection  with  project  team  members  about  progress. 

Review  of 

Implementation 

Process 

There  were  two  distinct  phases  in  the  project:  Phase  1 related  to  the 
preliminary  identification  of  the  population  and  the  development  of  the 
survey  instrument.  Corporate  data  were  used  to  develop  the 
demographic  profile  and  validate  the  population  under  study.  Ethical 
review  was  required  for  this  study.  A literature  review  was  conducted  of 
ED  in  urban  multicultural  environments  to  assist  in  development  of  the 
survey  instrument.  A variety  of  other  information  was  gathered  also  to 
assist  in  the  development  of  the  survey  instrument  including  interviews 
and  meetings  with  providers  (ED  staff  and  family  physicians),  reference 
to  other  experts,  meetings  with  community  representatives,  438  pre- 
development interviews  using  a mini-survey  with  people  visiting  the  ED. 
Of  these  people,  36.5%  agreed  to  participate  in  focus  groups  to  pre-test 
the  types  and  phrasing  of  questions  to  be  included  in  the  final  version  of 
the  survey  tool.  One  community  meeting  was  held  and  one  focus  group 
with  28  participants  provided  feedback.  The  final  instrument  was 
piloted. 

Telephone  interviews  were  then  conducted  in  Phase  2 of  the  study  with 
322  respondents  (first  telephone  interview),  and  320  completed  the 
follow-up  telephone  interview.  Removing  interviewees  who  did  not  meet 
selection  criteria  resulted  in  a final  sample  of  261 . 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Measurable,  specific,  relevant  goals. 

■ Clearly  defined  roles  of  project  team  members. 

■ Team  composition. 

■ Usefulness  of  stakeholder  involvement  to  provide  feedback  on 
tools. 

■ Benefits  of  allowing  sufficient  time  for  pre-testing  and  piloting  of  the 
survey  tool. 

Evaluation  of  Urban  Patients’  Choice  of  an  Emergency  Department 
as  their  first  Contact  with  Primary  Health  Care  Services 

(Evaluation  Project) 
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■ Use  of  a database  shell  that  has  application  beyond  project. 

■ Early  planning  for  dissemination  of  project  results. 

■ Usefulness  of  the  role  of  and  Independent  Evaluator. 

Challenges 

■ Length  of  time  to  conduct  telephone  interviews  (pre  and  follow  up). 

■ A portable  computer  was  needed  for  the  interviewee  to  conduct 
interviews  (in  multiple  languages). 

■ Lack  of  appropriate  venue  for  ethical  review  for  project. 

■ Reaching  the  local  community  for  timely  and  meaningful 
involvement  in  focus  groups. 

■ Complexity  of  telephone  interview  tool  led  to  less  time  for  data 
analysis.  Too  few  resources  were  dedicated  to  data  analysis. 

■ Outcome  of  database  is  still  in  beta  version. 

■ Difficulty  obtaining  CRHA  data. 

Results 

Intended  Project  Outcomes 

■ Outcome  #1  - development  of  a transferable  tool  was 
accomplished.  It  is  in  a beta  stage  of  development. 

■ Outcome  #2  - demographic  profile  of  the  patient  population  was 
prepared. 

■ Outcome  #3  and  #4  - data  are  available  on  access  and  barriers. 

Unintended  Project  Outcomes 

■ The  “Wall-Mart  Greeter”  effect  on  ED  patients. 

■ Extensive  branching  database  will  be  useful  in  other  environments. 

Innovations  and  Unique  Outcomes 

■ Exceptional  quality  of  individual  team  members. 

■ Ease  of  partnership  with  ED  administration,  staff,  and  physicians. 

■ Unique  survey  tool/database. 

■ Opportunity  for  innovation  - Emergency  Waiting  Room  “greeters.” 

■ Opportunity  for  innovation  - revised  ethical  review  process. 

■ Project  head  liaised  with  ED  administration  and  head  physician 
from  outset. 

■ 96%  of  users  spoke  fluent  English,  6%  Gurjrati,  3%  Punjabi,  9%  (16 
other  languages). 

■ The  majority  of  non-Canadian  ancestors  originated  in  Europe. 

■ Most  visits  occurred  between  8:00  AM  and  5:00  PM  (74%),  and 
between  8:00  AM  and  noon  (43%). 

Survey  results  indicated  that 

■ people  make  reasonable  choices  about  where  to  seek  care; 

■ a greater  proportion  of  participants  reported  that  they  would  attend 
the  ED  for  increasingly  urgent  situations  (actual  usage  patterns 
support  this  finding); 

■ nearly  twice  as  many  patients  with  more  resource-intense 
conditions  attempted  to  contact  their  family  physician  prior  to 
visiting  the  ED  than  patients  with  less  resource-intense  conditions; 

■ most  patients  presenting  at  the  ED  were  not  high  users  of  the  ED 
for  primary  care  (only  2%  indicated  regular  use  of  the  ED); 

■ a significant  proportion  of  participants  attempted  to  contact  another 
medical  professional  prior  to  going  to  the  ED  (27%  a family 
physician,  14%  a walk-in  clinic).  Many  (61%)  consulted  a non- 
professional before  going  to  the  ED; 

■ patient  worry  and  sense  of  urgency  drive  the  majority  of  patients  to 
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the  ED  for  primary  care  despite  other  options; 

■ ED  populations  are  idiosyncratic  (e.g.,  proximity  to  the  ED  did  not 
influence  visiting  the  ED;  convenience  was  rarely  cited  as  a reason 
for  going  to  the  ED  as  was  wanting  to  see  a specialist;  very  few 
participants  reported  being  told  by  a physician's  office  to  go  to  the 
ED).  (See  Project  Report  for  information  on  trends  and  statistically 
significant  differences  between  people  whose  mother  tongue  was 
English  and  not  English); 

■ 96%  of  participants  spoke  English  (75%  reported  being  born  in 
Canada,  nearly  80%  reported  that  their  ancestors  had  come  from 
North  America  or  Europe); 

■ Vietnamese  and  Chinese  may  have  been  under-represented  in  the 
sample;  and 

■ given  the  participation  rate,  it  is  feasible  to  gather  evidence  to  make 

decisions  concerning  primary  care  in  the  busy  environment  of  the 
ED.  !i 

Learning 

■ Perceptions  of  some  multicultural  communities  that  the  ED  is  the 

appropriate  primary  health  care  site.  ! 

■ It  is  important  to  follow  up  distribution  of  project  findings  with 
community  stakeholders. 

■ It  may  be  useful  to  translate  the  database  shell  itself  into  some  of 
the  other  major  languages  used  by  the  ED  population. 

Contribution  to  National  Dimensions 

Access 

■ The  project  did  not  measure  changes  in  access.  It  gathered  baseline 
data  on  the  demographic  profile  of  the  multicultural  population  of  the 
Peter  Lougheed  Centre’s  Emergency  Department,  why  users  accessed 
the  ED  for  primary  care  services,  and  what  if  any,  barriers  to  alternatives 
users  encountered. 

■ Users  typically  accessed  the  ED  for  what  they  thought  were  urgent 
reasons.  Patient  worry  and  sense  of  urgency  drive  users  to  the  ED. 

Users  typically  consulted  their  family  physician,  or  alternate  medical 
professional  before  going  to  the  ED. 

■ Statistically  significant  differences  were  reported  for  those  participants 

whose  mother  tongue  was  not  English  who  said  they  would  go  to  the  ED 
for  treatment  of  a minor  injury  (cut  finger)  than  those  whose  mother  j 

tongue  was  English. 

■ Trends  indicated  that  multicultural  groups  whose  mother  tongue  was  not 
English  were  more  likely  to  use  the  ED  for  non-urgent  situations  than 
were  people  whose  mother  tongue  was  English. 

Quality 

Integration 

■ The  integration  of  the  team  unit  was  an  important  aspect  of  the  project’s 
success. 

■ Integrating  the  key  stakeholders,  ED  staff,  and  community  members 
contributed  significantly  to  the  effective  functioning  of  the  project  and  the 
ultimate  achievement  of  project  goals. 

Health  Status 

Cost- 

Effectiveness 

Transferability 

" The  database  tool. 

■ The  team  composition  and  process. 

■ Method  of  involving  stakeholders. 
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■ Use  of  an  Independent  Evaluator. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

Need  for  an  appropriate  ethical  review  process. 
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Project  Name 

Evaluation  of  a Primary  Health  Care  Clinic  According  to  the  Primary 
Health  Care  Parameters  of  First  Contact,  Longitudinality, 
Comprehensiveness  and  Coordination  - UCMC  Sunridge  Project 

(Evaluation  Project) 

Project 

Description 

This  evaluation  project  was  conducted  in  Calgary  Regional  Health  Authority 
to  establish  an  evaluation  process  within  the  tenets  of  primary  care  and 
creation  of  a general  methodology  for  evaluating  community-based 
practice.  The  aim  of  the  project  was  to  access  basic  data  available  from  a 
variety  of  sources,  develop  a process  for  the  measurement  of  the  basic 
tenets  of  primary  care  (first  contact/access,  longitudinality, 
comprehensiveness,  and  coordination);  review  asthma  care  as  an  indicator 
of  quality  of  care;  and  identify  key  success  factors  for  strategic 
development  of  a primary  health  care  evaluation  model  for  UCMC- 
Sunridge.  UCMC-Sunridge  is  a community-based  family  practice  teaching 
clinic  mandated  to  provide  patient  care  as  well  as  serve  as  a training  site 
for  family  medicine  residents.  A main  goal  of  this  project  was  to  incorporate 
the  evaluation  findings  into  a process  of  continuous  service  evaluation  for 
the  clinic  and  to  disseminate  the  process  of  evaluation  to  other  primary 
care  clinics.  Asthma  was  chosen  as  an  area  for  review  of  content  of 
practice  because  of  a readily  available  validated  tool  and  knowledge  of 
incidence  of  asthma  as  a chronic  disease  in  the  general  population. 

Expected 

Outcomes 

Project  Rationale 

■ “As  medicine  becomes  increasingly  evidence-based  and  health  care 
increasingly  outcome  oriented,  an  evaluation  process  for  primary  care 
becomes  essential.  The  current  environment,  which  is  basing  the 
reform  of  the  health  care  system  on  primary  care,  requires  a framework 
to  produce  the  evidence  to  measure  this  reform”  (Project  Report). 

Goals 

■ Develop  an  evaluation  system  for  a primary  health  care  clinic  that  is 
aligned  with  four  domains:  first  contact,  longitudinality, 
comprehensiveness  and  coordination. 

■ Demonstrate  methods  of  assessing  care  delivery  processes  to  defined 
groups  of  patients  in  a clinic  setting. 

■ Review  the  information  system  needs  to  accommodate  and  support 
clinical  management,  clinical  operations  and  the  proposed  evaluation 
system. 

■ Begin  implementation  of  the  Continuous  Quality  Improvement  (CQI) 
process. 

■ Identify  key  success  factors  and  barriers  to  implementing  an  evaluation 
system  in  a primary  health  care  setting. 

Initial  and  Intermediate  Outcomes 

■ UCMC-Sunridge  has  conducted  a preliminary  assessment  of  their  level 
of  care. 

■ A CQI  process  was  initiated  into  which  staff  have  had  input. 

■ Key  stakeholders  have  had  an  opportunity  to  rank  primary  care  quality 
indicators. 

■ A manual  is  prepared  that  can  be  used  by  other  family  practice  medical 
centres  who  want  to  evaluate  their  services  and/or  begin  a CQI 
process. 

■ A new  software  system  has  been  installed  and  data  have  been  cleaned 
and  verified. 
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Lona-term  Outcomes 

■ Improve  and/or  maintain  level  of  quality  care  at  Sunridge. 

■ Through  dissemination  of  a manual,  improve  and/or  maintain  the  level 
of  quality  of  care  at  other  family  medical  centers/physicians’  offices. 

■ Reduce  the  amount  of  background  work  required  by  family  medical 
centers  to  incorporate  evaluation/CQI  processes. 

■ Establishment  of  an  evaluation  process  within  the  tenets  of  primary 
care  and  creation  of  a general  methodology  for  evaluating  community- 
based  practice. 

Methodology  of 
the  Independent 
Evaluation 

■ Due  to  the  unique  nature  of  this  project,  the  independent  evaluation 
was  viewed  as  a meta-evaluation  (i.e.  evaluating  an  evaluation). 

■ The  independent  evaluators  implemented  the  following  model  to  guide 
this  meta-evaluation:  reviewing  the  design  and  offering 
recommendations  for  strengthening  it;  providing  technical  assistance 
during  and  at  the  end  of  the  evaluation;  and  reviewing  evaluation 
procedures,  findings,  and  reports. 

■ The  independent  evaluators  also  used  a logic  model  to  help  frame  the 
evaluation. 

■ Input  was  gathered  from  the  community  care  provider,  patients, 
community  members  (random  sample),  nurses  and  physicians  at 
UCMC-Sunridge. 

Review  of 

Implementation 

Process 

Primarv  care  indicators 

■ The  project  team  obtained  input  on  the  most  important  indicators  of 
primary  care  (within  the  domains  of  first  contact,  longitudinally, 
comprehensiveness,  and  coordination  in  mind)  and  the  best  way  to 
measure  each  of  them.  A fifth  concept  (patient  physician  relationship) 
was  added.  To  address  quality  in  quality  management,  several  models 
were  referenced  to  begin  the  development  of  indicators.  Starfield’s 
work,  Safran’s  PCAS  tool  and  the  Canadian  Council  on  Health 
Services  Accreditation  (acceptability,  effectiveness,  accessibility, 
safety,  efficiency,  appropriateness  and  competence),  Clinical  Value 
Compass,  CRHA  balanced  scorecard,  CMA  Logic  model  used  to  guide 
evaluating  innovations  in  primary  care.  Also  involved  the  patient  and 
their  perspectives  of  quality  care.  A 4-step  quality  improvement  process 
was  used  (developing  teams  and  teamwork;  having  access  to  tools  or 
developing  tools;  understanding  the  quality  improvement  cycle, 
communicating  results  to  all). 

■ An  extensive  literature  review  on  indicators  was  conducted  using 
Medline,  PubMed,  Health  Star,  Psyclnfo. 

■ A rating  scale  to  select  appropriate  indicators  was  developed  by  the 
project  team. 

■ Indicators  were  ranked  by  physicians,  nurses,  other  health  care 
providers,  patients,  and  non-patient  members  of  community.  Patients 
and  community  members  participated  in  focus  groups  to  acquire  more 
in-depth  information/opinions. 

■ Workshops  were  held  with  physicians/nurses  to  decide  on  feasibility 
and  most  appropriate  measurement  methods. 

■ The  Primary  Care  Assessment  Survey  (PCAS)  was  chosen  as  one  of 
the  measurement  tools.  It  measures  dimensions  of  primary  care, 
possesses  good  psychometric  properties,  is  used  in  the  US  and  Britain, 
and  provides  benchmarking  data  from  comparative  purposes  (based  on 
subjective  reporting  by  patients).  The  Access  Instrument  and 
Comprehensiveness  Instrument  was  developed  to  account  for 
important  indicators  that  did  not  fit  into  the  PCAS. 
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Asthma  care 

■ The  project  team  carried  out  a review  of  asthma  patients  as  a pilot  test 
for  gathering  health  status  information  on  a prevalent  chronic  disease. 

Information  technoloov  assessment 

■ The  project  team  contracted  the  services  of  an  information  systems 
company  to  evaluate  potential  software  for  a primary  care  clinic  that 
could  incorporate/facilitate  the  CQI  program.  One  system  was  chosen 
and  installed. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Qualitative  findings  revealed  good  communication  among  team 
members  and  the  importance  of  immediate  feedback. 

Challenges 

■ Unexpected  system  changes  and  delays  occurred  due  to  problems  with 
ethical  and  scientific  review;  time  constraints  experienced  by  UCMC- 
Sunridge  physicians;  misunderstandings  regarding  the  goals  of  the  IT 
report  which  resulted  in  a substantial  rewrite;  challenges  in  obtaining 
information  from  Alberta  Health  and  Wellness;  and  the  work  overload  of 
one  team  member. 

■ Physicians  and  nurses  complained  of  the  time  it  took  to  collect  data. 

Results 

With  an  on-site  project  champion,  active  involvement  of  clinic  staff,  and 
support  from  uniguely  skilled  individuals  in  methodology,  literature  review, 
focus  group  facilitation,  data  manipulation/collation,  financial  planning  and 
management,  the  project  developed  an  evaluation  process  based  on 
principles  of  primary  health  care,  tools,  and  a costing  analysis  to  increase 
the  capacity  of  an  average  practitioner  to  develop  an  evaluation  system 
within  for  own  practice. 

Learning 

To  conduct  guality  assessments  the  following  is  reguired: 

■ Accurate  and  timely  data  that  can  be  converted  into  useful  information. 

■ Analytical  capacity  to  compile  information. 

■ Guidelines,  standards  of  excellence  to  access  priorities  for 
improvement. 

■ The  literature  review  revealed  that  cost  information  on  family  practice 
clinics  is  non-existent. 

Contribution  to  National  Dimensions 

Access 

■ Of  the  original  33  indicators  of  access  to  care,  8 were  chosen  for 
inclusion  in  the  final  CQI  evaluation  template. 

■ The  information  gathered  from  these  indicators  is  now  available  to 
improve  patient  access  to  UCMC-Sundridge.  These  indicators  include 
waiting  time,  availability,  utilization  of  services  and  demographics. 

Quality 

■ 71  guality  indicators  were  identified  and  37  were  subseguently  included 
in  the  final  CQI  evaluation  template.  Indicators  include  continuity, 
comprehensiveness,  patient  physician  relationship  and  satisfaction. 

■ Clinical  guality  was  assessed  through  the  asthma  care  chart  audit. 

■ Data  gathered  for  these  indicators  provided  planners,  physicians  and 
other  staff  with  the  ability  to  gauge  the  status  of  these  indicators  over 
time  and  make  changes  to  health  care  based  on  the  results. 

Integration 

■ The  information  system  selected  for  use  in  the  evaluation  of  clinics  has 
the  ability  to  integrate  primary  and  secondary  services,  Alberta  Health 
and  Wellness  billing  data,  and  laboratory  reports,  all  of  which  are 
important  areas  of  the  health  care  system. 

■ 20  indicators  of  integration  were  identified  and  5 were  chosen  for 
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inclusion  in  the  final  CQI  evaluation  template. 

■ Analysis  of  the  information  gathered  over  time  will  aid  in  the  integration 
of  external  services  with  those  provided  by  UCMC-Sunridge. 

Health  Status 

■ Data  gathered  through  the  CQI  evaluation  process  developed  at  UCMC- 
Sunridge  offers  specific  health  status  information  (e.g.,  pap  smear  rates, 
mammogram  rates). 

■ The  asthma  chart  review  process  contributed  to  knowledge  of  the  health 
status  of  UCMC-Sunridge  patients. 

Cost- 

Effectiveness 

■ The  project  assessed  the  costing  of  data  collection  and  analysis 
including  use  of  existing  data  sources,  the  primary  care  assessment 
survey,  the  access  tool,  and  the  comprehensiveness  tool. 

Transferability 

There  are  2 new  categories  of  products  that  are  transferable  from  this 
project:  process  and  tools. 

■ Process:  A manual  has  been  produced  that  is  suitable  for  use  bv  other 
primary  care  clinics/offices  for  implementing  an  information  technology 
evaluation  system. 

■ Tools:  The  list  of  50  indicators  and  the  instruments  based  on  these  will 
be  invaluable  to  others  interested  in  primary  care  evaluation.  These  tools 
consist  of  primary  care  assessment  survey-modified,  the  access 
instrument,  and  the  comprehensiveness  instrument. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

This  project  has  contributed  a list  of  indicators  related  to  access,  quality  and 
integration  for  evaluating  family  practice  clinics. 
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Project  Name 

Enhancing  Palliative  Care  for  Cancer  Patients  (CRHA) 

(Demonstration  Project) 

Project  Description 

This  demonstration  project  consisted  of  four  initiatives  aimed  at 
improving  the  quality  of  care  for  palliative  cancer  patients.  The  first 
initiative  involved  the  creation  and  evaluation  of  a Quality  Improvement 
Process  to  develop  and  apply  criteria  for  the  selection  of  projects.  The 
other  three  initiatives  focused  on  improving  palliative  care  in  specific 
areas. 

Specifically,  the  Education  and  Needs  Assessment  initiative  involved 
assessment  of  needs  of  family  physicians,  community  care 
coordinators,  oncologists,  patients,  and  families  followed  by  the 
development,  dissemination,  and  evaluation  of  a Palliative  Care  (PC) 
brochure.  The  Communication  initiative  involved  the  development, 
implementation  and  evaluation  of  a standardized  consultation  note 
format,  or  template,  from  the  Tom  Baker  Cancer  Centre  (Calgary, 
Alberta)  to  family  physicians.  The  third  initiative  involved  development 
and  implementation  of  a Clinical  Practice  Guideline  for  Palliative 
Sedation  including  its  evaluation  to  assess  changes  in  knowledge  and 
practice. 

Expected  Outcomes 

Expected  results  of  the  Quality  Improvement  Process  (QIP) 

1 . Identification  of  projects  that  would  result  in  the  enhancement  of 

palliative  care  for  cancer  patients.  : 

2.  Determination  and  application  of  criteria  for  project  prioritization. 

3.  Refinement  of  project  ideas  and  development  of  clearly  defined 
project  statements  for  each  project. 

4.  Implementation  of  projects  that  required  strictly  operational 
decisions. 

5.  Identification  of  project  team  members  including  assignment  of 
tasks  to  design,  implement,  and  evaluate  project  activities. 

6.  Assurance  that  the  project  would  contribute  to  improved  quality  of 
care  by  ongoing  communication  with  the  project  team. 

7.  Sponsorship  of  the  project  by  reporting  on  results  and  advocating 
for  adoption  of  recommendations. 

For  the  Education  and  Needs  Assessment  Initiative 

1 . Development  of  an  educational  brochure  on  palliative  care  services 
in  CRHA. 

2.  Assessment  of  patients’/families’  knowledge  and  attitudes  of 
palliative  care  prior  to  and  following  implementation  of  an 
educational  brochure  on  palliative  care  philosophy  and  local 
resources. 

3.  Assessment  of  the  education  and  information  needs  of  patients, 
patients’  families,  oncologists,  family  physicians,  community  care 
coordinators,  and  other  health  providers,  so  that  future  initiatives 
could  be  developed  to  ease  patients’  transition  to  palliative  care. 

For  the  Communication  Initiative 

1 . Development  of  a standardized  communication  template  and 
process  for  oncologists,  family  physicians  and  home  care 
coordinators. 

2.  Evaluation  of  the  benefits  of  a standardized  consultation  format/ 
letter  (including  diagnosis,  prognosis,  treatment  plan,  role  and 
expectations  of  the  family  physician  and  TBCC  follow  up)  sent  by 
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the  TBCC  to  patients’  family  physicians  after  initial  and  subsequent 
visits  to  the  TBCC. 

3.  Evaluation  of  the  benefits  of  a dedicated  phone  line  at  the  TBCC  to 
facilitate  access  to  primary  care  practitioners  and  oncologists 
regarding  mutual  patients,  including  requests  for  lab  reports  and 
other  information  about  patients. 

For  the  Palliative  Sedation  initiative 

1 . Education  of  care  providers  in  the  appropriate  use  of  Palliative 
Sedation  in  the  treatment  of  intractable  symptoms. 

2.  Updated  Palliative  Sedation  Clinical  Practice  Guideline. 

3.  Information  about  level  of  appropriate  use  and  adoption  of  Palliative 
Sedation  Clinical  Practice  Guideline  in  the  treatment  of  intractable 
symptoms. 

Methodology 

Project  Implementation 

For  QIP 

■ Facilitated  focus  group  discussion. 

■ Priority  setting  exercise  (valuing  exercise). 

For  Education  and  Needs  Assessment 

■ Literature  review  of  educational  preparation  and  needs  of  palliative 
patients/families. 

■ Forced-choice  and  open-ended  survey  questionnaire  for  family 
physicians,  oncologists,  home  care  coordinators,  patients/families 
to  assess  their  own  and  others’  education/training  needs. 

■ Participative  development  of  survey  questionnaire  (forced-choice 
and  open-ended)  by  patients  and  families  for  patients/families  to 
assess  the  success  of  the  brochure. 

■ Dissemination  of  brochure  for  3-month  period. 

■ Content  analysis  of  qualitative  data,  descriptive  statistics  of 
quantitative  data. 

For  Communication  initiative 

■ Literature  review  of  standardized  communication  processes  and 
standardized  reporting  formats. 

■ Creation  of  draft  template,  review  by  family  physicians,  and  4- 
month  pilot-testing  period  by  oncologists. 

■ Three  rounds  of  dissemination  of  the  finalized  template. 

■ Pre-post-surveys  to  family  physicians  who  had  seen  patients  in  the 
Lung  Clinic  over  a 4-month  period  prior  to  distribution  of  the 
template. 

■ Post-survey  of  family  physicians’  ratings  of  the  template. 

■ Focus  group  with  oncologists  to  assess  satisfaction  of  use  of  the 
template. 

For  Palliative  Sedation  initiative 

■ Literature  review. 

■ Development  of  Palliative  Sedation  CPG  and  12  sample  scenarios 
of  3 different  types  for  inclusion  in  the  pre-post  survey 
questionnaire. 

■ Dissemination  of  CPG  for  a one-year  period. 

■ Retrospective  chart  audit  (pre-test  data)  and  4 months  following 
dissemination  of  CPG  (post-test  data). 

■ Review  of  appropriateness  of  sedation  by  palliative  experts. 
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Independent  Evaluation 

1 . Process  evaluation  through 

■ Review  of  project  documents, 

■ Attendance  at  Leadership  Team  meetings, 

■ Interviews  with  Leadership  Team  members, 

■ Regular  communication  with  Research  Methodologist  and 
Research  Assistant. 

2.  Interviews  with  Project  Director. 

3.  Focus  groups  with  Leadership  Team. 

4.  Verification  of  themes  and  findings  by  Project  Director  and 
Research  Methodologist. 

Review  of 

Implementation 

Process 

■ Leadership  Team  formed  to  oversee  all  four  initiatives.  Each  | 

initiative  created  its  own  sub-team. 

■ Ethics  approval  was  granted  through  the  Conjoint  Scientific  Review 
Committee  (Calgary  Regional  Health  Authority  and  University  of 
Calgary). 

For  QIP 

■ A care  map  and  flow  chart  of  coordinated  care  were  developed  for 
palliative  patients  including  a facilitated  focus  group  process  to 
establish  appropriate  selection  criteria  for  the  prioritization  and 
selection  of  projects  (although  three  initiatives  had  already  been 
chosen  for  the  HTF  project). 

For  Education/Needs  Assessment 

■ A needs  assessment  survey  questionnaire  was  developed  based 
on  a review  of  the  literature  of  assessment  of  needs  of  palliative 
patients,  families  and  providers,  as  well  as  primary  data  collection 
from  palliative  patients,  and  home  care  coordinators,  and 
distributed  (via  mail-out)  to  oncologists,  family  physicians  and  home 
care  coordinators. 

■ An  educational  brochure  was  developed  based  on  input  from 
patients  and  families.  A needs  assessment  questionnaire  included 
(pre-test)  questions  about  the  brochure.  Post-test  questions 
involved  brief  mail-out  survey  to  patients  and  family  members. 

For  Communication  initiative 

■ A literature  review  of  standardized  communication  processes  and 
standardized  reporting  formats. 

■ Creation  of  draft  template,  review  by  10  family  physicians,  and 
piloting  by  oncologists  at  the  Lung  Clinic. 

■ Dissemination  of  template  to  oncologists  (through  meeting  with 
them  to  explain  the  nature  of  the  research  and  encourage  use  of 
the  template). 

» Pre-post-surveys  distributed  to  family  physicians  who  had  seen 
patients  in  the  Lung  Clinic  over  a 4-month  period  prior  to 
distribution  of  the  template  to  evaluate  its  use  and  assess 
communication  issues  with  oncologists. 

■ Low  rate  of  use  by  oncologists  in  first  iteration  led  to  second 
distribution  of  template  and  second  survey  to  family  physicians. 

■ Due  to  low  rate  of  use  by  family  physicians  who  had  treated 
palliative  cancer  patients  and  a third  distribution  of  the  template 
was  undertaken. 

■ Post-survey  of  family  physicians’  ratings  of  the  template. 

■ Focus  group  with  oncologists  to  assess  satisfaction  of  use  of  the  j 
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template  and  its  value. 

For  Palliative  Sedation  initiative 

■ Ethics  review. 

■ Literature  review  of  current  practices  in  palliative  sedation, 
dissemination  of  CPGs,  and  evaluation  of  the  effectiveness  of 
CPGs. 

■ Development  of  Palliative  Sedation  CPG  by  Sedation  CPG  Team. 

■ Development  of  12  sample  scenarios  of  3 different  types  for 
inclusion  in  the  pre-post  survey  questionnaire.  These  were  pilot 
tested  by  a group  of  volunteer  family  physicians  in  Edmonton. 

■ Scenarios  revised. 

■ Dissemination  of  CPG  for  a one-year  period  through  presentation, 
mail-out,  grand  rounds,  conference. 

■ Pre-post  questionnaires  distributed  to  family  physicians, 
oncologists,  community  care  coordinators  to  assess  knowledge  and 
practice  of  palliative  sedation  following  dissemination  of  the  CPG, 
pre-post  assessments  of  changes  in  level  of  knowledge  through 
survey;  changes  in  practice  were  measured  through  self-reported 
descriptions  of  past  episodes. 

■ Retrospective  chart  audit  conducted  on  patients  4 months  prior  to 
dissemination  of  CPG  (pre-test  data)  and  4 months  following 
dissemination  of  CPG  (post-test  data)  to  assess  administration  of 
medication  in  last  48  hours  of  life. 

■ Final  review  of  appropriateness  of  sedation  conducted  by  Team 
Leader  and  3 other  palliative  experts. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Composition  of  Project  teams  is  appropriate  and  efficient  since 
there  are  content  experts  on  each  project  and  a coordinating 
Leadership  Team  that  is  comprised  of  members  of  each  project. 

■ Project  Director  reports  directly  to  senior  administration  facilitating 
communication  with/and  support  from  senior  administration. 

■ Research  Methodologist  has  been  instrumental  in  research  design, 
instrument  design  and  development,  review  and  synthesis  of 
literature  and  data. 

■ Personal  communication  of  Project  Director  and  other  Leadership 
Team  members  with  oncologists  and  family  physicians  has 
increased  support  for  project. 

Challenges 

■ Ethics  review  process  through  CRHA/U  of  C delayed  the  project  by 
2 months. 

■ Data  collection  with  palliative  patients  and  families  is  difficult 
(emotional  issues). 

■ Participation  of  sufficient  numbers  of  oncologists  in  use  of  the 
communication  template. 

■ Sensitivity  around  Palliative  Sedation  CPGs  (often  confused  with 
euthanasia). 

■ Time  to  conduct  chart  audits  to  assess  appropriate  administration 
of  sedation. 

■ Uptake  of  CPG  by  practitioners. 

■ Brochures  that  are  used  by  patients/families. 

Results 

Qualitv  Improvement  Process 

■ Composition  of  multi-disciplinary  Leadership  Team  is  appropriate 
for  a Quality  Improvement  Process. 
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■ Focusing  on  do-able  projects  is  important  to  maintain  the  vitality  of 
the  process. 

Education  and  Needs  Assessment  Initiative 

■ Brochure  reached  26.5%  of  patients/families. 

Aggregated  results  of  the  needs/satisfaction  assessment  indicate 
priority  needs  as  follows: 

For  Family  Physicians 

Symptom  control  other  than  pain 
Alternative/complementary  medicine 
Local  palliative  resources 
For  Oncologists 

Symptom  control  other  than  pain 
Understanding  cultural  beliefs  and  values 
Stress  management  for  caregivers 
For  Community  Care  Coordinators 

Alternative/complementary  medicine  therapy 
Grief  counseling  for  survivors 
Understanding  cultural  beliefs  and  values 
For  Patients 

Alternative/complementary  medicine  therapy 
Information  about  pain  and  its  treatment 
Information  about  diagnosis/prognosis 
For  Families 

Information  about  local  palliative  care  resources 
Information  about  diagnosis/prognosis 
Information  about  pain  and  its  treatment 

Communication  Initiative 

■ Comparison  of  results  with  baseline  data  indicated  that  family 
physicians  who  had  received  the  standardized  consultation  note 
considered  the  template  an  improvement  over  previous  reporting 
formats.  It  allowed  for  easy  access  to  relevant  and  timely 
information  (although  the  Research  Methodologist  cautioned  that 
change  in  methodology  may  have  had  some  influence  on  positive 
results). 

■ Focus  group  discussion  between  the  Communication  Team  Leader 
and  oncologists  revealed  that,  from  the  perspective  of  oncologists, 
the  template  had  achieved  a)  increased  timeliness  of  dictations 
received  by  family  physicians  and  b)  increased  organization  of  the 
content  of  letters  included  in  patients’  files. 

■ Anecdotal  feedback  obtained  from  nurses  at  the  Lung  Clinic 
supported  this  finding  in  that  nurses  said  they  found  transcribing 
oncologists’  dictations  relatively  easy  using  the  template  format. 

■ Focus  group  information  also  indicated  that  oncologists  enjoyed 
using  the  template,  and  while  they  used  it  for  palliative  patients 
only,  they  said  they  would  use  it  for  all  patients  in  the  Lung  Clinic 
beginning  in  the  fall  of  2000.  Oncologists  also  expressed  a desire  to 
use  the  template  in  other  Clinics  as  well  (e.g.,  Neurology  Clinic). 
They  suggested  that  the  template  could  be  tailored  for  more 
specific  use  (e.g.,  for  follow  up,  clinical  trials),  and  that  it  could  be 
used  with  non-palliative  patients.  They  felt  that  some  clarification  of 
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use  of  terms  was  required,  and  emphasized  the  need  to  maintain 
patient  identifiers. 

■ With  respect  to  assessment  of  family  physicians’  satisfaction  of 
telephone  communication  with  oncologists,  family  physicians 
reported  most  satisfaction  with  the  amount  of  information  received 
once  appropriate  contact  was  made.  However,  they  did  report 
difficulty  making  that  appropriate  contact.  Physicians  offered 
several  suggestions  for  improved  telephone  communication — most 
of  which  pointed  to  a dedicated  phone  line — one  of  the  original 
objectives  identified  by  the  Communication  Team.  (See  Final 
Project  Report  for  list  of  suggestions.) 

■ Oncologists  suggested  that  project  results  be  disseminated  at 
grand  rounds  at  the  TBCC,  to  senior  administrators  at  the  TBCC, 
and  through  the  Community  Cancer  Network  Newsletter  that  is 
circulated  across  regions. 

Palliative  Sedation  Initiative 

■ Pre-post  test  results  indicated  slight  increases  (not  significant)  in 
knowledge  about  appropriate  sedation  across  oncologists,  family 
physicians  and  community  care  coordinators.  However,  among  the 
different  scenarios  it  appeared  that  delirium  as  a condition 
influenced  how  providers  responded  to  the  scenario.  This  pointed 
to  the  need  for  further  investigation. 

■ Results  of  the  chart  audit  indicated  a higher  rate  of  appropriately 
sedated  patients  after  dissemination  (not  statistically  significant). 
However,  the  rate  of  inappropriately  sedated  patients  remained 
high  (pre  and  post  dissemination).  Data  suggested  that  the 
management  of  delirium  may  be  a significant  factor  that  deserves 
further  study. 

■ Significant  change  was  reported  in  family  involvement  and  team 
members’  involvement  in  care  decisions. 

Learning 

■ Creative  presentation  formats  of  resource  directories/information 
including  dissemination  strategies  are  required  to  reach  palliative 
patients/families. 

■ Patients/families  and  providers  (i.e.,  oncologists,  family  physicians, 
community  care  coordinators)  have  differing  ideas  about  education 
and  training  needs  of  each  other. 

■ Families  think  the  level  of  interpersonal  skill  of  oncologists  could  be 
improved  with  respect  to  imparting  diagnostic  information  to 
patients  and  general  information  about  palliative  care  to  family 
members. 

■ Families  want  and  expect  information  about 
alternative/complimentary  therapies. 

■ Further  research  is  required  on  interdisciplinary  approaches  to 
educate  health  professionals  including  team  building  and  cross- 
cultural  training  to  facilitate  increased  understanding  of  each  other’s 
needs. 

■ Citizens  need  knowledge  about  the  philosophy  of  palliative  care 
and  its  resources  in  order  to  make  informed  decisions  about  their 
care. 

■ Patients  need  to  have  increased  confidence  that  family  physicians 
have  access  to  timely  information  about  patient  care  related  to 
palliation. 

■ Satisfaction  rates  of  timeliness  of  receipt  of  information  by  family 
physicians  indicates  that  the  standardized  communication  template 
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should  be  used  for  non-palliative  as  well  as  palliative  patients. 

■ Post-test  data  will  be  collected  on  use  of  the  template  in  other 
settings  beyond  the  Tom  Baker  Cancer  Centre. 

■ The  study  emphasized  the  need  for  a clinically  integrated  database 

where  on-line  real  time  information  is  simultaneously  available  to  ' 

home  care,  family  practitioner’s  office  and  oncologists.  Structures 
and  processes  to  support  such  a database  do  not  currently  exit. 

■ The  Sedation  team  felt  that  the  process  used  to  develop  CPGs  was 
useful  and  will  be  applied  to  the  development  of  other  CPGs. 

■ The  project  team  understood  the  challenges  associated  with 
improving  the  uptake  of  CPGs.  Alternative  methods  of 
dissemination  (that  is,  in  addition  to  mail-outs)  were  used  and  found 
to  be  successful  (e.g.,  discussions,  grand  rounds).  Mail-out 
strategies  are  considered  another  useful  mechanism.  Palliative 
clinicians  plan  to  incorporate  the  CPG  into  relevant  consults,  and 
distribute  the  CPG  to  members  of  the  care  team  at  clinically 
appropriate  times. 

■ Expert  reviewers  were  concerned  at  the  high  rate  of  inappropriate 
sedation  at  both  pre  and  post-test.  Further  analysis  on  the  reasons 
for  this  rate  may  be  conducted.  Preliminary  analysis  suggests  that 
delirium  may  have  played  a role  in  almost  60%  of  all  cases 
reviewed. 

■ The  Sedation  Team  reinforced  the  importance  of  continuing  to 
advocate  at  the  national  level  “that  the  philosophy  of  palliative  care 
and  appropriately  utilized  interventions  such  as  palliative  care  are 
ethical  and  legal”  (Final  Project  Report). 

■ The  360-degree  assessment  of  satisfaction  and  perspectives  of 
other  provider  groups  creates  clear  direction  for  a long-term 
strategy  to  improve  communication  and  service  related  to  palliative 
care. 

■ Evaluation  helps  to  maintain  a focus  on  evidence-based  palliative 
care  practice. 

Contribution  to  National  Dimensions 

Access 

■ The  brochure  facilitated  access  to  information  by  patients  and  families. 
However,  use  of  information  about  palliative  care  resources  may  be 
influenced  by  readiness  levels  and/or  interest  of  patients  and  their 
families  in  specific  information  at  certain  points  of  their  illness. 
Distribution  of  the  Palliative  Care  brochure  to  a large  number  of  a variety 
of  caregivers  in  this  study,  and  receipt  by  a very  small  number  of 
palliative  patients  and  their  families,  points  to  the  need  to  develop 
creative  ways  to  attract  people’s  attention — particularly  when  patients 
and  families  struggle  to  remain  hopeful.  Print-based  media  may  be 
effective  when  used  in  combination  with  other  innovative  presentation 
formats/techniques. 

■ Access  to  diagnostic  and  treatment  information  from  oncologists  by 
family  physicians  and  other  caregivers  will  help  the  transfer  of  patients 
across  levels  of  care. 

Quality 

■ The  360-degree  assessment  mechanism  used  to  assess  multiple 
perspectives  of  provider  groups  and  patients/families  was  a good 
strategy  that  led  to  increased  knowledge  about  patients’  and  providers’ 
needs  for  information,  training,  and  education.  This  was  a useful 
mechanism  to  support  further  planning  processes  aimed  at  improving 
quality  of  care. 
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■ The  Quality  Improvement  Process  (and  its  criteria  for  selection  of 
appropriate  projects)  is  a good  mechanism  to  improve  the  quality  of 
services  offered. 

■ The  standardized  consultation  note  template  was  well  received  by 
oncologists  and  family  physicians  in  that  it  created  a mechanism  that  led 
to  timely  transfer  of  information  between  oncologists  and  family 
physicians,  and  ultimately  to  smoother  transfer  of  palliative  patients 
between  curative  and  palliative  care  environments.  Use  of  the  template 
with  non-palliative  audiences  in  multiple  settings  attests  to  its  broad 
application. 

■ Use  of  the  tools  to  disseminate  and  assess  knowledge  levels,  past  and 
current  practice  related  to  palliative  sedation,  in  themselves,  helped  to 
raise  awareness  and  knowledge  of  the  Clinical  Practice  Guideline  for 
Palliative  Sedation.  They  also  raised  consciousness  about  the  need  for 
adoption  of  the  CPG  for  improved  palliative  care. 

■ Results  of  the  chart  audit  indicated  a higher  rate  of  appropriately  sedated 
patients  after  dissemination  (not  statistically  significant).  However,  the 
rate  of  inappropriately  sedated  patients  remained  high  (pre  and  post 
dissemination).  Data  suggested  that  the  management  of  delirium  may  be 
a significant  factor  that  deserves  further  study.  Significant  change  was 
reported  in  family  involvement  and  team  members’  involvement  in  care 
decisions.  Anecdotal  information  suggests  increased  dialogue  about 
palliative  sedation  and  increased  requests  for  information  for  pain  and 
symptom  management.  Together,  these  signs  indicate  movement 
towards  improved  quality  of  care  for  palliative  patients. 

Integration 

■ In  this  project,  three  initiatives  were  explored  that  encourage  and  support 
integrated  delivery  of  palliative  care  services.  Knowledge  about 
education,  training,  and  communication  needs  of  family  physicians, 
oncologists,  community  care  coordinators,  patients,  and  families  is  a first 
step  to  learning  what  providers  need  to  know  about  providing  quality 
care  to  palliative  patients  and  their  families,  and  a first  step  to 
understanding  what  it  might  take  to  work  successfully  together.  The 
needs  assessment  process,  therefore,  proved  to  be  a good  technique  to 
begin  moving  toward  successful  integration  of  services. 

■ The  Leadership  Team  learned  that  structures  and  processes  need  to  be 
in  place  to  support  integrated  delivery.  Family  physicians  must  be 
connected  to  oncologists  in  timely  and  effective  ways.  Sharing  of 
electronic  health  records,  for  instance,  may  more  effectively  and 
efficiently  supplement  and/or  replace  print-based  information.  Issues  of 
privacy,  of  course,  will  need  to  be  addressed. 

■ The  Leadership  Team,  itself,  was  a successful  example  of  multiple 
provider  groups  working  across  sectors  and  systems  to  improve 
palliative  care.  The  Team  demonstrated  that  a common  vision,  purpose, 
and  commitment  go  a long  way  to  change  how  services  are  provided. 
Requests  from  other  providers  to  examine  and  explore  the  model  of 
integrated  service  delivery  employed  in  this  project  encourage  the 
Leadership  Team  that  they  are  on  the  right  track. 

Health  Status 

Cost- 

Effectiveness 

Transferability 

■ The  Quality  Improvement  Process  (and  its  criteria  for  selection  of 
appropriate  projects)  is  a good  mechanism  to  improve  the  quality  of 
services  offered. 

■ The  assessment  tools  and  data  collection  instruments  used  in  this 
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project  can  form  the  basis  for  assessing  needs  and  perspectives  of 
multiple  provider  groups.  The  addition  of  a Research  Methodologist  in 
the  early  stages  of  project  design  and  implementation  is  important  to 
draw  as  much  learning  and  evidence  from  projects  as  possible. 

■ The  Palliative  Care  brochure  is  a print-based  resource  for  palliative 

services  in  the  CRHA.  Others  may  use  it  to  guide  the  development  of 
their  own  resource  directories.  j 

■ The  list  of  education/training  needs  identified  by  each  stakeholder  group 
for  themselves  and  for  other  care  providers  may  be  of  benefit  to  others. 

■ The  consultation  template  is  available  for  applied  use  in  palliative 
environments.  Others  in  this  project  expect  that  it  has  applicability  in 
non-palliative  environments  as  well. 

" The  Palliative  Sedation  Guideline  (and  the  CPG  related  to  delirium  that 
is  currently  being  developed)  are  transferable  across  care  environments. 

■ The  integrated  model  of  service  delivery  for  palliative  care  is  one  that 
may  be  applicable  to  other  similar  environments. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

— 

Policy 

Implications 

■ To  facilitate  access  to  primary  health  care  service  (as  first  point  of 
contact  with  the  health  system),  better  strategies  need  to  be  developed 
to  reach  palliative  patients  and  their  families.  Information  dissemination 
through  brochures  is  marginally  useful. 

■ Participative  processes  as  those  used  in  the  creation  of  the  standardized 
communication  format  (oncologists  to  family  physicians)  help  to  create  a 
mechanism  tool  that  will  be  used  by  multiple  providers. 

■ To  encourage  and  facilitate  inter-disciplinary  collaboration  a 360-degree 
assessment  of  perceptions  (in  this  case  with  respect  to  education  and 
training  needs)  is  useful  before  making  decisions  about 
information/program  development  training  needs. 

■ Structures  and  processes  are  required  to  support  inter-disciplinary 
collaboration  (e.g.,  information  sharing  through  electronic  real-time 
health  records). 

■ It  is  important  to  make  decisions  with  the  involvement  of  each 
stakeholder  group  involved  in  providing  palliative  care  including  the 
patient/family. 

■ Palliative  Sedation  CPG  needs  broader  dissemination  and  evaluation  of 
its  adoption. 

■ The  CPG  related  to  delirium  requires  development  (and  is  underway). 

■ Standardized  reporting  formats  (easily  accessible  electronically)  are 
critical  to  facilitating  communication  between  care  providers — in  this 
case  between  oncologists  and  family  physicians.  However,  standardized 
templates  may  be  useful  across  multiple  disciplines.  These  need  to  be 
co-developed  and  piloted. 

■ Policies  may  need  to  be  developed  in  relation  to  standardized 

documentation  on  charts  for  palliative  patients  since  a project  finding 
indicated  difficulty  in  discerning  the  degree  to  which  appropriate 
medications  had  been  administered.  j 

■ Patients  and  families  expect  that  family  physicians  will  provide 
information  about  alternative/complementary  medicine.  This  has 
implications  with  respect  to  training  of  medical  students. 

■ Periodic  chart  reviews  may  need  to  be  conducted  to  assess  the  extent  to 
which  CPGs  are  followed. 
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Implications  for 
Research 

■ The  addition  of  a Research  Methodologist  in  the  early  stages  of  a project 
is  beneficial. 

■ Palliative  sedation  CPG  is  available  along  with  a method  to  test  its 
adoption  and  appropriate  usage. 

■ Data  collection  from  palliative  patients  and  their  families  must  be  as 
unobtrusive  as  possible.  Pre-post  designs  are  generally  not  relevant. 

Implications  for 
Evaluation 

■ Linkages  between  research/evaluation  and  improved  patient  outcomes 
must  be  clear  to  physicians  in  order  to  encourage  their  involvement  in 
data  collection  and  recruitment  of  patients  (i.e.,  once  oncologists 
understood  that  the  template  was  appreciated  by  family  physicians  and 
ultimately  benefited  the  patient,  they  were  more  willing  to  be  involved). 

■ Evaluation  should  result  in  evidence-based  practice  information  for  care 
providers. 

■ Evaluation  processes,  can  in  themselves,  increase  awareness  of  CPGs. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
Community  Health  Centre  Models 


Project  Name 

Airdrie  Regional  Health  Centre 

(Evaluation  Project) 

Project  Description 

The  goal  of  the  Airdrie  evaluation  project  was  to  monitor  the 
implementation  process  and  the  first  year  of  operation  of  the  new 
Health  Centre.  Airdrie  Regional  Health  Centre  (ARHC)  was  planned 
and  constructed  as  a collaborative  project  by  the  Calgary  Regional 
Health  Authority  (CRHA),  Rocky  View  Municipal  District  No.44  and  the 
City  of  Airdrie.  In  collaboration  with  other  community  organizations,  a 
multi-use  community  health  facility  was  conceptualized.  The  Centre 
was  officially  opened  in  November  1998.  At  present,  CRHA  services 
offered  at  the  Centre  include  health  promotion,  education,  counseling, 
disease/illness  prevention,  home  care,  speech/language  assessment 
and  treatment,  and  hearing  assessments. 

Expected  Outcomes 

Objectives  of  the  Internal  Evaluation 

1.  Document  the  planning,  implementation,  and  evaluation  processes 
of  the  Centre  from  conception  to  the  end  of  the  first  year  of 
operation. 

2.  Identify  structures  and  processes  that  facilitated  or  inhibited  the 
development  of  the  Centre. 

3.  Establish  the  Centre’s  vision,  principles,  and  service  model  to  meet 
the  health  needs  of  Airdrie  and  the  surrounding  region. 

4.  Assess  various  aspects  of  Centre  operations. 

Objectives  of  the  Independent  Evaluation 

1 . To  review  and  analyze  findings  according  to  the  national 
dimensions  of  inquiry. 

2.  To  review  and  analyze  the  planning,  implementation,  and 
evaluation  processes  according  to  the  principles  of  primary  health 
care. 

Methodology 

Project  Implementation 

■ A descriptive,  process  evaluation  approach  was  used  to  document 
the  planning  and  implementation  of  the  new  Centre  and  its  first  year 
of  operation.  The  local  (internal)  project  evaluation  and  independent 
evaluation  objectives  were  inter-related  and  pursued 
collaboratively. 

Independent  Evaluation 

■ Data  collection  consisted  of  three  main  activities:  review  of 
available  documentation,  direct  observation  and  recording,  and 
face-to-face  interviews  with  key  stakeholders. 

Review  of 

Implementation 

Process 

■ A document  review  was  carried  out  to  clarify  the  context  in  which 
the  ARHC  was  developed  and  to  accurately  describe  the  planning 
and  implementation  processes. 

■ The  local  project  coordinator  attended  more  than  25  meetings  over 
the  duration  of  the  project.  Proceedings  of  the  meetings  were 
systematically  recorders  using  an  observation  tool  to  document 
issues,  perspectives,  and  the  nature  and  duration  of  discussions. 

■ Individual  and  group  face-to-face  interviews  were  conducted  with 
key  stakeholders.  Personal  perspectives  were  solicited  to  achieve 
the  following  objectives: 

■ Obtain  information  to  accurately  describe  the  conceptualization, 

Airdrie  Regional  Health  Centre 

(Evaluation  Project) 
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planning,  and  implementation  of  the  Centre; 

■ Determine  what  factors  facilitated  and  inhibited 
conceptualization,  planning,  implementation  and  service  model 
development; 

■ Determine  the  extent  to  which  the  planning,  implementation 
and  evaluation  processes  were  guided  by,  and  demonstrated, 
primary  health  care  principles; 

■ Ascertain  the  different  perspectives  and  forces  at  play  during 
planning  and  implementation; 

■ Determine  what  factors  inhibited  and  facilitated  the  ability  to 
apply  primary  health  care  principles; 

■ Determine  how  key  stakeholders  perceive  the  ARHC  model 
affects  access  to,  quality  and  integration  of  health  services; 

■ Determine  what  lessons  were  learned  through  the 
conceptualization,  planning  and  implementation,  processes  that 
might  be  useful  for  other  jurisdictions;  and 

■ Determine  what  lessons  were  learned  through  the  HTF 
evaluation  project. 

■ Two  sets  of  interviews  were  carried  out.  The  focus  of  the  first  set  of 
interviews  was  on  the  conceptualization,  planning  and 
implementation  of  ARHC  and  the  perspectives  of  informants  on 
primary  health  care  and  its  implementation  within  the  CRHA.  In  the 
latter  set  of  interviews  questions  pertained  to  the  later  stage  of  the 
implementation,  ongoing  Centre  operations,  and  establishment  of 
the  service  delivery  model. 

" Interview  questions  were  drafted  collaboratively  by  the  independent 
evaluator  and  local  project  coordinator.  The  questions  served  as  a 
guideline  for  the  interview  sessions. 

■ Narrative  information  obtained  through  interviews  was  introduced 
into  NUD*IST,  a computer  package  designed  for  storage  and 
indexing  of  qualitative  data. 

■ The  qualitative  data  were  reviewed  independently  by  the  local 
project  coordinator  and  independent  evaluator. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ A well-established  network  of  health  and  social  service  agencies, 
with  strong  relationships  at  the  front  line. 

■ Open  and  transparent  planning  processes  with  community 
involvement  and  positive  media  coverage,  in  the  early  stages  of 
development. 

■ In  the  early  planning  stages,  joint  CRHA  and  municipal 
collaboration  in  achieving  a common  vision  for  a multi-use 
community  health  centre  exemplifying  primary  health  care 
principles. 

Challenges 

■ Minimal  preparatory  work  to  prepare  proposal. 

■ Lack  of  clarity  around  the  concepts  of  primary  health  care. 

■ Timing  of  evaluation  (lack  of  clear  vision  and  service  model  not 
established,  therefore  limited  ability  to  identify  and  evaluate). 

■ Confusion  regarding  roles  and  expectations. 

■ CRHA  contextual  changes  (CRHA  and  senior  management 
changes). 

Planning/Development  Issues 

■ Divergence  in  community  and  physician  perspective  on  the  need 
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for  on-site  and  after  hours  medical  services.  The  public  perceived 
that  after  hours  medical  services  should  be  enhanced,  some 
members  perceived  that  Airdrie  had  a right  to  a 24-hour  treatment 
facility,  and  there  was  frustration  that  early  planning  envisioning  a 
comprehensive  health  centre  was  not  acted  upon.  Physicians, 
however,  opposed  after-hours  coverage  and  perceived  any 
provision  of  medical  services  out  of  the  Centre  as  direct  competition 
to  their  services.  ! 

■ Financial  obstacles — that  is,  lack  of  proof  of  the  cost-effectiveness 
of  primary  care  models,  and  lack  of  identification  of  appropriate 
provider  payment  mechanisms  to  support  the  vision  and  principles 
of  primary  health  care. 

■ Lack  of  a guiding  framework  (primary  care  versus  primary  health 

care  or  medical/provider  perspective  versus  community-driven 
model).  |j 

■ Organizational  instability. 

■ Communication  issues  related  to  confusion  between  willingness  of 
region  to  support  after-hours  and  medical  services. 

■ Sense  of  non-inclusion  by  community  stakeholders  and  lack  of 
recognition  of  Airdrie  as  a unique  rural  community. 

Results 

■ The  HTF  Project  Director,  in  consultation  with  the  Airdrie  Regional 
Health  Centre  (ARHC)  development  team,  developed  an  Airdrie 
Regional  Health  Centre  Service  Delivery  Model  framework. 

Centre  Planning  and  Implementation 

■ A holistic  primary  health  approach  to  planning,  including 
consideration  of  Airdrie’s  unique  service  needs,  was  evident  in  the 
early  stages  of  Centre  planning.  However,  once  it  became  evident 
that  public  funding  would  not  support  the  vision  of  an  innovative, 
comprehensive  multi-use  health  center,  the  focus  of  planning 
shifted  solely  to  planning  a physical  facility  to  replace  the 
inadequate  public  health  site,  with  provision  for  community 
agencies  as  facility  tenants.  No  further  effort  was  made  to  develop 
a Centre  vision  and  service  model  within  the  planning  stages  of  the 
Centre  development.  This  aspect  of  planning  was  to  be 
reintroduced  during  the  first  year  of  Centre  operation. 

Planning/Development  (Positives) 

■ Collaboration/co-location  of  services  facilitated  improved 
communication,  cooperation,  and  reduced  duplication  of  effort  and 
simplified  referral  processes  among  programs  housed  at  the 
ARHC.  Also  perceived  to  result  in  increased  convenience  (access) 
for  clients  and  increased  efficient  service  delivery. 

■ Building  features  (design). 

■ Location  of  Centre  (increased  access,  visibility,  increased  walk-in 
traffic  particularly  for  family  services). 

■ Broader  clientele  attracted  (e.g.,  Hutterites  more  likely  to  visit). 

■ Increased  collaboration  and  willingness  to  collaborate  among  staff 
(e.g.,  with  community  and  outreach  workers). 

Planning/Development  (Detractors) 

■ Organizational  management  structure  (i.e.,  different  reporting 
structures,  budgets,  geographical  catchment  areas)  did  not 
encourage  integrated  service  delivery. 

■ Fear  of  losing  control  of  program  areas  that  may  lead  to  inequitable 
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access  to  core  services  if  decisions  are  left  to  local  Centre  and 
area. 

■ Lack  of  coterminous  catchment  areas  among  programs  offered 
through  the  Centre. 

■ Limited  range  of  services  perceived  by  community  members  who 
advocated  for  additional  or  enhanced  medical  services,  mental 
health,  rehab,  prevention  and  support  services  for  teens,  seniors 
and  caregivers.  Professional  staff  acknowledged  that  services  did 
not  meet  public  expectations,  but  that  mechanisms  were  needed  to 
enhance  public  awareness  about  existing  services  and  the 
development  of  better  partnerships  with  existing  local  family 
physicians. 

■ Transportation  to  the  Centre  (some  was  offered  to  Calgary). 

■ No  common  client  information  system  exacerbated  with  uncommon 
boundaries.  Present  database  systems  were  inconducive  to 
addressing  service  impact  or  program  effectiveness. 

■ Slow  progress/intangible  outcomes. 

■ Structural  issues  with  the  building  (e.g.,  location  of  common  coffee 
areas  not  amenable  to  integration). 

Learning 

Based  on  the  results  of  the  ARHA  evaluation,  it  appears  substantially 
more  difficult  to  implement  primary  health  care  using  a health  centre 
approach  within  a regionalized  system  that  as  a stand-alone  entity  such 
as  a community  health  centres  are  traditionally  known.  A number  of 
underlying  reasons  are  suggested: 

■ While  individual  centres  have  the  ability  to  readily  define  their 
centre  vision,  objectives,  target  populations,  and  operational 
structures,  implementing  such  a centre  within  the  context  of  a 
health  system  necessitates  having  to  negotiate/define  a common 
vision  and  direction  amid  conflicting  interests  and  perspectives; 

■ The  size  and  complexity  of  the  regional  organization  structure  may 
limit  the  attention  and  priority  given  by  senior  decision-makers, 
resulting  in  a sustained  level  of  inertia; 

■ Historical  organizational  structures  along  functional  program  or 
discipline  lines  are  not  consistent  with  the  primary  health 
care/community  health  approach  to  population  or  community-based 
planning; 

■ Risk  aversion  around  the  concepts  of  community  involvement  and 
community  accountability  may  be  more  evident; 

■ There  may  be  some  discomfort  with  the  concept  of  community 
accountability  especially  in  the  early  stages  of  development  of  the 
regional  health  authority;  and 

■ The  study  revealed  a tension  between  the  primary  health  care 
principle  of  community  empowerment  and  the  CRHA’s  decision  to 
pursue  partnerships  with  independent  health  service  providers. 

Issues/Challenaes 

■ Long  debate  on  primary  versus  primary  health  care  without 
resolution  or  action. 

■ Community  perception  of  having  significant  input  into  the  direction 
and  vision  of  the  Centre  which  was  not  supported  by  the  region. 

■ No  explicit  definition  of  primary  health  care,  and  lack  of  clearly 
defined  role  of  family  physicians  regarded  as  a fundamental  barrier 
to  achieving  a primary  health  care  vision. 

■ Need  to  prove  cost-effectiveness  prior  to  securing  investment  in 
primary  health  care  models. 
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■ Reluctance  of  government  to  investigate  alternate  forms  of  funding 
(e.g.,  private  sector)  of  multi-use  centers. 

■ Lack  of  regional  support  for  significant  community  involvement 
including  that  of  family  physicians.  This  risk  aversion  strategy 
(contrary  to  the  primary  health  care  principle  of  community 
empowerment  and  intersectoral  collaboration)  served  to  inhibit 
service  model  development. 

■ Organizational  changes  within  the  region  led  to  competing 
priorities,  cutbacks  (regionally/provincially),  downsizing,  the 
necessity  to  forge  new  relationships,  and  develop  new  policies. 

■ Lack  of  focus  on  prevention/promotion  as  opposed  to  urgent  care. 

■ Lack  of  a defined  population  from  which  to  base  decisions  about 
needs  and  services. 

Facilitators  included 

■ Strategic  direction  of  the  CRHA  which  targeted  primary  health  care 
as  the  priority  organizational  change  as  well  as  a provincial  and 
national  focus  on  primary  health  care.  A regional  visioning 
document  entitled  Primary  Health  Care  Focused  System  was 
developed  as  a reference  point  in  ARHC  service  delivery  model 
development  process.  The  creation  of  a Primary  Health  Care 
Development  Leader  position  also  made  more  tangible  the  concept 
of  primary  health  care  (translated  in  the  vision  into  reality).  This 
demonstrated  support  from  senior  management  within  Community 
Health  Resources. 

■ A changed  mindset  among  mid-managers  within  Community  Health 
Resources  was  regarded  as  a breakthrough  in  that  status  quo  was 
no  longer  regarded  as  desirable. 

■ Positive  working  relations  between  front-line  staff  and  community 
agencies  facilitated  collaborative  planning  as  evident  in 
development  of  a common  vision  and  plan  for  private  financing  of  a 
multi-use  comprehensive  Centre. 

■ Co-location  of  services  contributed  to  improved  communication, 
improved  access  for  clients  and  intersectoral  collaboration  as  a 
step  toward  integration. 

Contribution  to  National  Dimensions 

■ There  is  strong  public  expectation/demand  for  access  to  medical 
services  within  a health  centre  such  as  the  Airdrie  Regional  Health 
Centre  (ARHC).  Public  sentiment  suggests  the  term  “Health  Centre” 
connotes  the  provision  of  medical  and  support  services,  including 
laboratory  and  medical  imaging. 

■ The  CRHA  was  unable  to  dispel  the  general  community  perception  of 
need  for  enhanced  medical  services. 

■ Access  to  the  Health  Centre  programs  was  perceived  to  be  greatly 
enhanced  by  its  central  and  visible  location  on  Airdrie’s  main 
thoroughfare,  close  proximity  to  the  major  shopping  area  and  large  free 
parking  area.  More  walk-in  traffic  was  noted  than  was  experienced  in  the 
previous  facility. 

■ Co-location  of  CRHA  staff  with  partner  agencies  was  reported  to  result  in 
easier  referral  processes  and,  hence,  greater  number  of  referrals  to 
Centre  programs. 

■ The  availability  of  community  meeting  facilities  was  reported  to  enhance 
individual  client  and  community  group  access  to  educational  sessions 
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and  presentations. 

Quality 

Integration 

■ Co-location  of  service  providers,  regardless  of  organizational  affiliation, 
was  perceived  to  contribute  to  greater  communication,  collaboration  and 
multidisciplinary  programming. 

■ Historical  functional  program  structures,  with  separate  budgets,  reporting 
structures  and  different  geographical  catchment  areas,  was  reported  to 
foster  continued  “stove-piping.”  The  historical  stove-pipes  were  found  to 
be  barriers  to  an  integrated  multi-disciplinary  approach  to  meeting 
community  needs. 

■ The  population  catchment  areas  differed  among  the  various  CRHA 
programs.  This  barrier  to  integration  presented  difficulties  in  defining  a 
common  target  population,  which  in  turn,  implied  the  need  for  developing 
a common  client  database. 

Health  Status 

Cost- 

Effectiveness 

Transferability 

The  following  recommendations  were  offered  to  other  jurisdictions 

contemplating  the  development  of  a health  centre  within  a regional  context: 

■ Ensure  a common  and  clear  vision. 

■ Adopt  a position  in  the  primary  care/primary  health  care  debate. 

■ Be  tangible,  not  conceptual  and  keep  things  simple. 

■ Build  on  existing  service  strengths  and  models  rather  than  building  new 
ones. 

■ Assign  a task  force  with  a mandate  to  start  with  a broad  focus,  then 
move  quickly  to  specific  tasks  and  addressing  specific,  tangible  issues 

■ Engage  a diverse  range  of  people. 

■ Work  from  a common  base,  find  and  build  upon  common  interests 

■ Focus  on  building  capacities. 

■ Focus  on  internal  integration  before  extending  out  to  encompass  players 
outside  the  organization. 

■ Be  willing  to  take  risks. 

■ Look  to  build  a system  of  services,  not  stand-alone  services. 

■ For  effective  consultation  with  or  involvement  of  the  community,  strong 
leadership  and  a will  to  consult  are  needed. 

■ For  community  partnerships  to  work,  open,  meaningful,  and  transparent 
processes  are  needed. 

■ When  embarking  on  community  consultation,  ensure  that  you  know 
which  questions  to  ask  and  what  feedback  you  want  from  the  community 

■ Determine  which  core/standard  services  will  be  provided. 

■ Be  aware  of  political  situations. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

■ The  project  reported  frustration  at  the  amount  of  effort  expended  within 
the  CRHA  in  debates  regarding  the  concept  of  primary  care  versus 
primary  health  care  (i.e.,  based  on  historical  perspectives  around  a 
medical  versus  community-driven  health  centre  model).  The  debates 
served  to  divide  rather  than  unify  people  towards  a common  vision.  The 
lack  of  an  explicit  definition  of  primary  health  care,  including  the  role  of 
family  physicians,  was  seen  by  some  key  informants  as  a fundamental 
barrier  in  achieving  a primary  health  care  vision  within  the  ARHC  project. 

■ The  document  Primary  Health  Care  Focused  System:  Primary  Care 

118 


Synopsis  of  Independent  Evaluation  Reports 


Providers  and  CRHA  in  Partnership  provided  the  first  evidence  of  a clear 
and  common  vision  for  primary  health  care  within  the  CRHA.  This  vision 
was  used  as  a reference  point  in  the  ARHC  service  delivery  model 
development  process. 

■ The  creation  of  a Primary  Health  Care  Development  Leader  position  was 
an  important  factor  in  moving  primary  health  care  forward  within 
Community  Health  Resources. 
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Project  Name 

8th  and  8th  Health  Centre 

(Evaluation  Project) 

Project  Description 

The  purpose  of  the  evaluation  project  was  to  review  operations  of  the 
8th  and  8th  Health  Centre  in  downtown  Calgary  (since  its  inception 
March  1997)  with  a view  to  identifying  strengths,  opportunities  for 
improvement,  what  has  been  learned  to  date,  and  to  make 
recommendations  to  support  ongoing  development  of  the  Centre.  The 
intent  was  to  facilitate  decision  making  regarding  future  services  at  8th 
and  8th  and  to  explore  the  potential  implications  of  subsequently 
developing  other  similar  Health  Centres  within  the  CRHA. 

Expected  Outcomes 

Goals  of  the  Centre 

1 . To  provide  client-focused,  accessible,  and  acceptable  services. 

2.  To  eliminate  professional  and  structural  barriers  to  the  provision  of 
appropriate,  responsive,  and  cost-effective  service  delivery. 

3.  To  eliminate  cultural  and  language  barriers  to  accessing  health 
services. 

4.  To  promote  high  quality  of  work  life  for  staff  and  a welcoming 
atmosphere  for  clients. 

5.  To  eliminate  political,  professional,  financial,  and  other  barriers  to 
the  provision  of  comprehensive,  cost-effective  services  addressing 
individual  and  community  needs. 

6.  To  provide  comprehensive  and  integrated  services  addressing 
individual  and  community  needs. 

Evaluation  Goals 

1 . To  explore  the  extent  to  which  the  Health  Centre  is  perceived  to  be 
attaining  intended  goals  and  objectives. 

2.  To  develop  a framework  that  can  be  used  to  monitor  the  activities 
of  the  Health  Centre  on  an  ongoing  basis. 

Evaluation  Objectives 

1 . Describe  who  is  being  seen  at  the  Centre. 

2.  Describe  why  clients  choose  the  Health  Centre  over  other  health 
facilities. 

3.  Determine  the  merits  of  providing  service  over  a 24-hour  period. 

4.  Describe  client  satisfaction  and  awareness  of  services  provided  by 
the  Health  Centre. 

5.  Describe  staff  satisfaction  with  service  delivery. 

6.  Describe  stakeholder  perception  of  services  provided  and  needs 
met  (or  not  met). 

7.  Describe  how  well  the  Centre  has  achieved  integration  and 
coordination  of  service. 

8.  Develop  a preliminary  framework  for  determination  of  the  cost- 
effectiveness  of  this  community-based  model  of  service  delivery. 

Methodology 

Project  Implementation 

The  project  involved  a formative  evaluation  expected  to  facilitate 
ongoing  Centre  development  and  improvement.  An  exploratory, 
descriptive  approach  was  adopted.  Data  were  obtained  from  a variety 
of  sources: 

■ Centre  databases  (client  and  service  descriptive  data  including 
demographic  and  diagnostic  information,  time  and  purpose  of  visits, 
and  which  program  was  accessed,  e.g.,  medical  care,  public  health 
or  mental  health). 

■ Process  mapping  (a  process  mapping  exercise  was  undertaken  to 
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document  the  flow  of  clients  through  the  Centre).  Interviews  were 
conducted  with  all  Centre  program  coordinators.  (Each  step  of 
client  process  was  illustrated  by  an  external  consultant  and  then 
validated  by  staff.) 

■ Community  telephone  survey.  A random  survey  (N=400)  to 
determine  community  awareness  of  and  perceptions  about  the 
Centre  included  15  core  downtown  communities  identified  as  the 
target  population.  This  survey  was  supplemented  by  face-to-face 
interviews  with  48  individuals  who  did  not  have  telephones  or  did 
not  speak  English. 

■ CRHA  and  community  agency  survey.  Mail  survey  distributed  to 
148  key  community  stakeholders;  33  individuals  responded  and 
follow-up  interviews  conducted  with  10  (face  to  face). 

■ Staff  survey  provided  to  all  8th  and  8th  staff  to  gather  their  opinions 
regarding  program  processes,  satisfaction  with  the  work 
environment,  and  suggestions  for  improvement  (37%  response 
rate).  Face-to-face  interviews  were  conducted  with  20  staff. 

■ Cost-effectiveness  analysis.  A health  economist  from  the  University 
of  Calgary  assisted  the  project  team  by  developing  and  applying  a 
methodology  for  examining  the  cost  of  medical  services  within  the 
Centre  across  time  of  day. 

■ Quantitative  data  were  compiled  into  Epi  Info  or  Access.  Qualitative 
interview  data  were  transcribed  verbatim,  then  coded  independently 
by  two  team  members  involved  in  conducting  the  interviews. 
Themes  were  identified  and  reported. 

Independent  Evaluation 

The  project  included  the  project  team  (internal  evaluation)  as  well  as  an 
independent  evaluator.  The  primary  role  of  the  independent  evaluator 
was  to  address  the  six  national  dimensions  of  inquiry.  The  role 
independent  evaluator  has  indicated  that  given  the  expertise  and  level 
of  involvement  (of  the  Advisory  Committee)  there  was  little  need  to 
provide  direction  or  additional  input. 

Review  of 

Implementation 

Process 

■ In  response  to  recommendations  from  the  Community  Health  Task 
Force  on  Primary  Care  and  Primary  Health  Care  Needs  of  Inner 
City  Calgary  (1996),  the  8th  and  8th  Health  Centre  opened  in  1997. 
Services  provided  included  immediate  primary  medical  care  (24 
hrs/day),  mental  health  services,  public  health  services,  home  care 
services,  and  community  development/liaison.  The  Health  Centre 
was  developed  with  intention  to  adopt  the  principles  of  primary 
health  care. 

■ In  1998  the  CRHA  successfully  applied  for  HTF  funding  to  evaluate 
key  aspects  of  the  8th  and  8th  service  delivery  model. 

■ The  project  included  an  Evaluation  Advisory  Committee  whose 
members  provided  support  and  direction  to  the  project.  Evaluation 
questions  were  clear  from  the  outset  and  served  to  frame  data 
collection,  analysis,  and  reporting. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Planning  for  the  8th  and  8th  Centre  Evaluation  had  commenced  prior 
to  the  announcement  of  the  HTF  initiative.  The  evaluation  project’s 
goals  and  design  were  quite  well  established  at  the  outset. 

■ Tremendous  support  provided  by  Advisory  Committee  members. 

Challenges  (identified  as  methodological  issues  by  the  evaluator) 

■ Current  routine  data  collection  does  not  incorporate  factors  to  j 
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facilitate  a holistic  assessment  of  the  individual  and  the  identifying 
of  underlying  health  determinants.  Some  staff  expressed  discomfort 
directly  asking  questions  related  to  socio-economic  status,  lifestyle, 
and  client  coping  abilities. 

■ Limitations  with  the  database  included  limited  data  elements, 
deficiencies  with  the  computer  system,  maintenance  of  duplicate 
manual  records,  and  slow  production  of  performance  reports. 

■ Sampling  techniques  (staff  tended  to  exclude  clients  with  mental 
illness  due  to  reliability  concerns). 

■ Low  response  rate  from  staff  and  community  stakeholders. 

Results 

Information  was  gathered  about  the  clientele  of  the  8tn  and  8tn  Health 

Centre. 

■ 50,000  individuals  accessed  the  Centre  between  March  1997  and 
September  1999;  51%  male,  49%  female;  30%  between  20-29 
years;  only  7%  over  age  60;  48%  clients  accessing  the  Centre  were 
residents  of  the  15  core  downtown  communities. 

■ Three  distinct  clients  groups  emerged:  a)  those  living  in  the  core 
downtown  communities,  b)  those  working,  but  not  living,  downtown, 
and  c)  those  living  and  working  outside  downtown. 

" Between  3 and  5%  of  the  clients  were  estimated  to  have  highly 
complex  health  care  needs  (e.g.,  homelessness,  mental  illness, 
substance  abuse,  Hep-C,  HIV,  diabetes). 

■ Most  common  reasons  were  location  near  home  or  work,  or 
convenience  (46%),  followed  by  referral  (20%),  previous 
experience  (14%),  acceptable  waiting  times  (7%),  the  fact  that  it 
was  open/required  no  Personal  Health  Number  (7%),  and  they  liked 
the  service/provider  or  centre  (3%). 

■ 54%  clients  reported  they  had  a family  physician,  46%  did  not. 
Residents  of  the  inner  city  area  were  more  likely  to  report  not 
having  a family  physician  than  those  not  in  the  inner  city  target 
area.  Pattern  of  use  for  clients  without  a family  physician  was 
consistent  with  that  of  clients  with  a family  physician  in  terms  of 
time  of  day  when  they  accessed  the  Centre. 

■ Clients  with  a family  physician  reported  more  service  events 
related  to  injuries/poisonings,  respiratory  problems,  and  certain 
perinatal  events.  Clients  without  a family  physician  represented 
more  service  events  related  to  skin  and  subcutaneous  tissue  and 
mental  disorders  than  clients  with  a family  physician. 

■ Clients  with  a family  physician  were  more  likely  to  be  transferred 
from  the  Centre  by  ambulance.  The  number  of  ambulance  transfers 
per  month  from  the  Centre  to  acute  care  facilities  more  than 
doubled  over  a 16-month  period  suggesting  an  increase  in  the 
acuity  of  the  caseload  and  possibly  increased  confidence  (of 
medical  community)  in  the  ability  of  Centre  staff  to  appropriately 
triage  clients  to,  or  away  from,  emergency  departments. 

Health  Centre  programs  included  medical  care,  mental  health,  care  in 

the  community  (home  care),  public  health  and  community  development. 

Study  findings  related  to  the  main  Centre  programs  are  summarized 

below. 

Medical  Care 

■ 68%  increase  in  service  volume  over  study  period  (from  86  to  145 
clients  per  day);  a three-fold  increase  in  actual  service  volumes 

■ compared  to  Centre’s  functional  plan  developed  in  1997;  after- 
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■ hours  services  increased  over  the  period  of  study,  daytime  activities 
began  to  level  off  in  the  second  year  of  operation;  population 
served  in  medical  care  area  included  adults  (75%),  children  (20%) 
and  seniors  (5%). 

■ Physician  staffing  was  increased  by  1 FTE  during  peak  period  of 
summer  1999,  although  this  could  not  be  sustained. 

Mental  Health 

■ Client  profiles  (in  mental  health)  varied  substantially  from  other 
Centre  services  in  terms  of  education,  income,  employment,  etc. 

■ Mental  health  services  accounted  for  4%  of  service  events. 

Care  in  the  communitv  (Home  Care) 

■ On-site  home  care  services  accounted  for  8%  of  all  service  events 
(increase  from  1%  initially),  attributed  to  the  introduction  of  an  on- 
site wound  clinic  in  October  1998. 

■ Home  care  provides  a greater  proportion  of  its  services  to  those 
over  age  64  than  do  any  other  Centre  services. 

Public  Health 

■ The  average  number  of  services  remained  constant  over  the  study 
period;  the  majority  of  public  health  clients  were  children  under  five. 

Learning 

Overall  Learning 

■ The  project  resulted  in  a much  clearer  understanding  of  the 
characteristics  of  the  clients  served  and  their  reasons  for  accessing 
the  Centre. 

■ Survey  results  provided  considerable  information  about  Centre 
strengths  and  potential  areas  for  improvement. 

Specific  learning  identified  by  members  of  the  Evaluation  Advisory 

Committee  included  the  following. 

■ The  community  needed  to  push  hard  for  24-hour  urgent  care 
services. 

■ The  burden  of  proof  (to  measure  longitudinal  health  status 
outcomes  and  ultimately  cost-effectiveness)  should  not  be  placed 
on  one  project  unless  the  system  looks  at  itself  with  the  same 
measures. 

■ The  demand/volume  of  clients  was  (higher  than)  unexpected  given 
early  projections.  This  poses  additional  questions  such  as,  how  can 
one  meet  the  demand  in  ways  that  are  consistent  with  continuity  of 
care?  Can  people  be  turned  away  from  any  of  the  organizations 
given  a regional  health  service  mandate? 

■ Continuity  of  care  is  compromised  as  the  medical  services  provided 
at  the  Centre  is  currently  focused  primarily  on  episodic  care. 
Physicians  are  not  able  to  establish  long-term  relationships. 

■ Physician  education  is  important  (to  ensure  that  clients  and 
decision-makers  in  the  region  do  not  think  clients  receive  second 
class  service  at  the  Centre). 

■ Human  resources  are  important  - need  physicians  not  only  capable 

of  dealing  with  life  threatening  problems  but  also  who  can  provide 
psycho-social  holistic  care  to  complex  clients.  The  Centre  would  be 
better  skilled  to  maximize  the  skills/use  of  the  average  nurse  (rather 
than  hiring  nurse  practitioners).  |j 
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Contribution  to  National  Dimensions 


Access 

■ Mental  disorders  account  for  about  8-10%  service  events  in  the  primary 
medical  care  area  reflecting  a rate  of  increase  growing  faster  than  the 
mental  health  program.  This  possibly  reflects  a growth  in  the  number  of 
individuals  with  mental  disorders  accessing  the  Centre  during  the  late 
evening  or  night  when  no  mental  health  staff  are  on  site. 

■ There  appears  to  be  a considerable  and  largely  unexplained  demand  for 
Centre  services  on  the  part  of  individuals  who  neither  reside  nor  work  in 
the  downtown  area.  CHCs  are  defined  by  geographic  boundaries  or  by 
“issues  of  common  unity."  This  study  suggests  that,  within  a regionalized 
health  system,  such  delineation  cannot  as  readily  be  made. 

■ Access  to  Centre  services  by  the  seniors  population  was  lower  than 
expected,  given  the  proportion  of  seniors  residing  in  the  downtown  area. 
The  local  project  team  commented  that  the  Centre  was  “not  intended  to 
provide  the  highly  technical  and  complex  pre-hospital  care  that  often 
leads  seniors  to  seek  care  from  an  acute  care  facility.”  In  view  of  the 
Long  Term  Care  Review  committee  report  this  finding  has  policy 
implications  for  developing  an  ideal  primary  health  care  model  for  older 
people. 

■ Access  to  the  Centre  by  recent  immigrants  was  less  than  anticipated.  It 
is  recommended  that  greater  efforts  be  made  towards  increasing 
awareness  of  Centre  services  among  new  immigrants  and  the  agencies 
that  serve  them. 

■ The  main  reasons  for  accessing  the  Centre,  as  reported  in  client 
surveys,  include  location,  convenience,  referral,  and  satisfaction  (i.e.. 
with  services,  waiting  times,  and  staff).  Another  factor  may  be  that 
almost  half  the  clients  reported  not  having  a family  physician. 

■ The  pattern  of  Centre  usage  by  clients  with  and  without  family  physicians 
was  found  to  be  highly  consistent.  This  implies  a potential  conflict 
between  the  goal  of  providing  convenient  access  to  services  and  that  of 
continuity  of  care  with  a primary  family  physician. 

Quality 

Satisfaction  with  Services  and  Providers 

■ The  majority  of  clients  were  highly  satisfied  with  services  (93%). 

■ Centre  improvements  as  recommended  by  clients  included  reducing  wait 
times,  improving  signage,  free  access  to  supplies  (e.g.,  medication)  and 
equipment  for  those  who  could  not  afford  them,  and  improved  flow  of 
patients  through  the  Centre. 

■ 87%  of  clients  served  by  mental  health  staff  in  1999  scored  50  or  greater 
on  the  Mental  Health  Diagnostic  (GAF)  Scale,  which  generally  suggests 
that  they  require  hospital  care.  These  findings  suggest  the  possibility  that 
hospitalization  may  have  been  averted  for  a significant  number  of  mental 
health  clients  accessing  Centre  services  and  that,  with  support,  these 
individuals  can  continue  to  be  served  in  the  community. 

Issues  Identified  bv  Staff 

■ Frustration  with  the  inability  to  actualize  a primary  health  care  approach 
to  practice,  given  the  high  service  volumes. 

■ The  governance  structure  of  the  Centre  was  seen  as  not  conducive  to 
meaningful  community  involvement  in  the  ongoing  operation  of  the 
Centre. 

■ The  focus  on  immediate  care  overwhelms  the  capacity  of  the  Centre. 

■ The  need  to  clearly  define  the  target  population  with  a focus  on  the 
vulnerable  or  marginalized. 
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■ Safety  concerns  for  clients  and  staff,  particularly  receptionists  exposed 
to  potential  violence  from  mentally  unstable  or  inebriated  individuals 

■ concern  that  the  appearance  of  some  clients  caused  other  clients  to 

seek  care  elsewhere.  jj 

■ Lack  of  privacy  in  the  reception  area. 

Integration 

Barriers 

■ Existing  reporting/organization  structures  and  different  program 
mandates/geographical  catchment  areas. 

■ Physician  reimbursement  mechanisms  with  incentives  for  volume  and 
potentially  affecting  time  available  for  case  consultations. 

■ Lack  of  clear  definitions  and  directions  regarding  primary  health  care 
within  the  CRHA  (inconsistency  and  no  incentive  to  develop  an 
integrated  delivery  system). 

■ Database  limitations  (limited  ability  to  track  the  extent  of  integration  that  i 
occurs). 

Facilitators 

■ Integration  was  facilitated  by  personal  relationships  among  staff  within 
the  Centre. 

■ The  skills  of  all  reception  and  clerical  staff  were  augmented  through 
cross  training  (to  address  narrowly  defined  job  functions  based  on 
program  area  and  to  create  an  administrative  support  team). 

■ Internal  communication  was  enhanced  via  “voice  mail  hotline”  (staff 
could  access  newsworthy  items  from  work  or  home  at  their  convenience. 

Health  Status 

Cost- 

Effectiveness 

■ The  team  undertook  to  develop  and  assess  the  robustness  of  a cost- 
effective  methodology  suitable  for  comparing  operations  across  work 
shifts.  The  ultimate  objective  was  to  apply  the  methodology  in  comparing 
the  performance  of  the  medical  care  component  of  the  Centre  with  that 
of  other  settings.  The  cost-effectiveness  study  suggested  that,  in  the 
early  stages  of  Centre  operations,  the  night  shift  was  less  cost-effective 
than  the  other  two  shifts.  As  waiting  times  increased  during  the  other 
shifts,  however,  and  partly  due  to  differential  trends  in  costs  and  patient 
loads,  the  night  shift  was  estimated  to  become  relatively  more  cost- 
effective.  The  methodology  was  judged  to  be  feasible  for  inter-provider, 
inter-facility  comparisons. 

■ Development  of  an  econometric  methodology  to  assess  cost  within  a 
primary  health  care  setting  are  promising  providing  the  logic  of  these 
approaches  is  not  presented  in  an  unduly  complex  way. 

Transferability 

■ The  econometric  methodology  developed  and  tested  could  be  applied  in 

comparing  the  cost  of  medical  services  delivered  across  varying  models 
of  primary  health  care.  i 

■ Recommendations  to  other  jurisdictions  contemplating  the  development 
of  a similar  health  centre  within  the  context  of  a large  regional  health 
authority  structure  include  the  following: 

■ Plan  for  more  than  one  Centre  in  the  city.  Location  is  important.  1 

Such  multi-purpose  Centres  should  be  built  as  alternatives  to  the 
emergency  department  as  the  majority  of  clients  can  be  treated  by  a 
primary  health  care  centre.  Each  Centre  would  be  geographically 
based  and  target  the  needs  of  their  particular  community. 

■ An  ideal  model  might  be  independent  private  practices  linked  to  a 
multi-centre  hub. 

■ The  urgent  care  function  should  not  over-ride  the  centre  function  of  ; 
providing  comprehensive,  holistic,  and  preventative  care. 

■ Be  explicit  about  the  target  population  (accurate  projections  about 
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the  population  base  and  defining  a maximum  are  required). 

■ Establish  a community-based  governance  of  the  Centre  (not  a 
community  board  but  a broader  public  organization  may  create 
better  accountability  and  credibility  for  the  medical  community). 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

Restricted  activities 

The  provision  of  urgent  care  (versus  emergency  care)  in  a primary  health 
care  setting  has  resulted  in  questions  regarding  what  is  emergency  medicine 
and  who  should  provide  it.  This  may  have  implications  under  the  Alberta 
Health  Professions  Act,  which  requires  each  health  profession  to  specify 
which  of  its  members  are  regulated  to  perform  a series  of  restricted 
activities.  There  may  be  a fine  line  between  emergency  and  urgent  situations 
requiring  many  of  the  same  procedures. 

24-hour  access 

Compared  to  the  Airdrie  Regional  Health  Centre  where  the  community 
lobbied,  unsuccessfully  for  24-hour  access  to  services,  the  8th  and  8th 
Advisory  Committee  members  concluded  that  the  community  needed  to 
push  hard  for  24-hour  access.  (Regional  reports  in  Airdrie  indicated  there 
was  no  service  utilization  need  for  24-hour  access  at  the  Centre,  despite 
community  desires.) 

Phvsician  reimbursement 

A proposal  was  submitted  to  the  Alternate  Payment  Plan  (APP)  Committee, 
Alberta  Health  and  Wellness  and  the  Alberta  Medical  Association  to  seek 
alternate  modes  of  funding  medical  services  at  the  Centre  with  a view  to 
addressing  barriers  to  the  delivery  of  comprehensive  interdisciplinary  primary 
care  to  highly  vulnerable  groups  in  the  inner  city.  (Results  of  the  application 
not  yet  known.) 

Implications  for 
Research  and 
Evaluation 

■ The  econometric  methodology  developed  and  tested  in  this  study  could 
be  applied  in  comparing  the  cost-effectiveness  of  medical  services 
delivered  across  varying  models  of  primary  health  care. 

■ This  project  included  an  evaluation  Advisory  Committee.  The 
independent  evaluator  indicated  that  the  many  hours  and  high  caliber  of 
input  contributed  by  the  members  in  providing  direction  to  the  project 
served  to  keep  the  evaluation  focused,  ensure  diverse  perspective  in 
data  interpretation  and  ultimately,  a better  product. 

■ The  project  also  illustrated  the  benefits  of  formative  evaluation,  as  a 
number  of  structural  and  practice  changes  were  introduced  throughout 
the  course  of  the  study  (as  a result  of  evaluation  findings).  For  example, 
a change  in  the  way  clients  are  approached  in  registration  or  “triage”  to 
facilitate  consideration  of  service  providers  other  than  physicians  as  first 
contact  for  clients.  To  address  database  issues,  the  evaluation 
coordinator  created  data  entry  forms  on  Microsoft  Access  (linked  to  the 
Centre’s  main  database)  to  eliminate  the  need  for  duplicate  reporting. 
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Project  Name 

Northeast  Community  Health  Centre  (NECHC) 

Project  Description 

Demonstration  Project 

The  demonstration  project  consisted  of  a team  of  service  providers 
initially  referred  to  as  the  “enhanced  service  team.”  This  multidisciplinary 
team  was  comprised  of  health  professionals  selected  to  meet  the 
diverse  needs  of  the  Northeast  community  including  a dietician,  school- 
age  occupational  therapist,  mental  health  occupational  therapist, 
addictions  counselor,  child  mental  health  therapist,  social  work,  and 
multicultural  health  broker.  Note:  The  enhanced  service  team  worked  in 
concert  with  the  primary  health  services  team  and  over  time  was  not 
known  as  the  enhanced  service  team  but  rather  part  of  the  overall  staff 
at  the  Centre.  The  primary  health  services  team  consisted  of  family 
physicians,  nurse  practitioner,  nurses,  and  other  health  professionals  at 
the  NECHC. 

Evaluation  Project 

The  evaluation  project  (referred  to  as  the  internal  evaluation)  consisted 
of  three  main  components:  implementation  evaluation,  formative 
evaluation,  and  evaluation  of  the  enhanced  service  team. 

Expected  Outcomes 

The  goal  of  the  NECHC  (a  new  facility  established  in  1999)  was  to 
provide  services  and  to  promote  partnerships  so  that  residents  of 
northeast  Edmonton  will  be  able  to  access  publicly-funded  health 
services  and  related  resources  in  their  own  community  to  improve  their 
health  and  well-being.  Representatives  from  the  Capital  Health  Authority 
were  successful  in  securing  funding  under  the  HTF  to  conduct  both  a 
demonstration  and  an  evaluation  project  at  the  Centre.  Services 
available  at  the  Centre  include  Family  Health,  Community  Health,  Child 
& Adolescent  Health,  Mental  Health,  Women’s  Health,  Seniors’  Health, 
Lab  and  Diagnostic  Imaging  Services,  Audiology,  Dietician,  Multicultural 
Health  Broker,  Community  Asthma  Care  Centre,  and  Emergency 
Services. 

The  independent  evaluation  team  established  a series  of  nine  intended 
project  outcomes  in  consultation  with  the  internal  evaluation  (project) 
team. 

1 . Development  of  a Formative  Evaluation  Framework 

2.  Implementation  of  a Formative  Evaluation  Framework 

3.  Delivery  of  Accessible  Services  to  the  Community 

4.  Delivery  of  Relevant/Appropriate  Services  to  the  Community 

5.  Delivery  of  Quality  Services  to  the  Community 

6.  Delivery  of  Integrated  Services  to  the  Community 

7.  Delivery  of  Cost-Effective  Services  to  the  Community 

8.  Partnerships  are  Evident  and  Working 

9.  Framework  to  Measure  Potential  Impact  on  Health  Status 

Methodology  of 

Independent 

Evaluation 

■ An  independent  evaluation  was  conducted  (review  of  the  internal 
evaluation  and  findings  according  to  the  national  dimensions  of 
inquiry).  Several  other  evaluations  were  occurring  at  the  Centre 
concurrently,  funded  by  various  other  organizations  including  Glaxo 
Wellcome. 

■ The  independent  evaluation  team  adopted  a collaborative  approach 
to  the  evaluation  project  and  maintained  close  communication  with 
the  internal  evaluators  throughout  the  course  of  the  project.  Regular 
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update  meetings  were  held  to  maintain  communication  regarding 
status  and  progress  of  the  HTF  evaluation  components. 

■ A predominantly  qualitative  paradigm  was  used  for  the  independent 
evaluation.  The  independent  evaluation  team  utilized  several  data 
collection  methods,  including  interviews,  focus  groups,  document 
reviews,  and  a secondary  review  of  internal  data.  Data  were 
reviewed  relative  to  the  independent  evaluation  plan.  Interviews  and 
focus  group  data  were  analyzed  according  to  predominant  themes. 
Summary  reports  were  constructed  and  sent  back  to  participants  for 
review  and  verification. 

Review  of 

Implementation 

Process 

■ Long-standing  community  support  for  improved  health  services,  as 
well  as  leadership  support  at  the  board  level. 

■ Initial  funding  submission  to  HTF  (May  1998). 

■ Construction  of  the  NECHC  facility  (Summer  1 998). 

■ Official  opening  of  phase  one  (January  1999). 

■ Recruitment  of  external  consultants  to  complete  the  Formative 
Evaluation  (resulted  in  establishment  of  the  Formative  Evaluation 
Team),  as  well  as  consultants  to  complete  “The  Story  of  the 
NECHC”  (i.e.,  the  implementation  evaluation). 

■ Official  opening  ceremonies  for  NECHC  to  coincide  with 
commencement  of  Emergency  Services  (September  1999). 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ Leadership  support  (organizational  and  board  levels). 

■ Strong  community  support  for  the  Centre  (previous  community 
consultations  predating  the  construction  of  Centre  by  over  ten 
years). 

■ Presence  of  on-site  evaluation  coordinator  (has  been  referred  to  as 
the  central  interpreter  service  between,  for  example,  front  line  staff 
and  research  personnel). 

■ The  initial  orientation  program  was  quite  lengthy  and  attempted  to 
involve  as  many  staff  as  possible  to  develop  a common  vision,  build 
relationships.  Ongoing  orientation  was  established  for  new  staff. 

Challenges 

■ There  was  some  initial  challenge  integrating  the  pre-existing 
Clareview  Public  Health  Unit  into  the  Northeast  Community  Health 
Centre  (e.g.,  challenges  integrating  existing  services  with  the  new 
conceptual  model  and  service  delivery  areas). 

■ The  NECHC  team  experienced  a great  deal  of  pressure  from 
multiple  stakeholders  in  terms  of  evaluation  and  multiple  evaluations 
occurring.  Staff  were  required  to  balance  participation  in  evaluation 
activities  with  heavy  service  workloads. 

■ Development  of  data  collection  instruments  and  processes 
particularly  early  in  the  evaluation  process.  The  regional  registration 
system  was  unable  to  provide  data  specific  enough  to  answer 
evaluation  questions.  The  research  assistant  later  linked  the  four 
main  databases  in  use  at  the  Centre. 

■ Many  staff  had  no  previous  experience  with  evaluation  and  needed 
time  to  understand  that  the  evaluation  was  concerned  with  how  to 
improve  operations  of  the  Centre  as  a whole,  rather  than  about  them 
personally. 

Results 

1 . Development  of  a Formative  Evaluation  Framework 
The  NECHC  internal  evaluation  team  developed  a formative  evaluation 
framework  for  the  HTF  project  based  on  31  evaluation  questions 
organized  according  to  access,  integration,  model  change,  and 
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satisfaction.  A longer-term  evaluation  framework  (3-to  5-year  plan)  was 
established  in  consultation  with  the  Department  of  Public  Health 
Sciences  (University  of  Alberta). 

2.  Implementation  of  a Formative  Evaluation  Framework 
The  Formative  Evaluation  Framework  was  implemented  by  the 
Formative  Evaluation  Team  (a  team  of  four  external  consultants  hired  by 
the  internal  evaluation  team).  Each  member  assumed  primary 
responsibility  for  collecting  data  on  each  of  the  four  categories  (access, 
integration,  model  change,  and  satisfaction).  The  implementation  of  the 
formative  evaluation  included  significant  input  from  the  Centre  staff  and 
administration.  The  evaluation  process  included  rapid  turnaround  of 
evaluation  findings,  presentation  to  staff,  discussions,  and  development 
of  action  strategies  based  on  formative  evaluation  findings. 

3.  Delivery  of  Accessible  Services  to  the  Community 

The  NECHC  engaged  in  a prioritization  of  service  delivery  to  ensure 
access  to  services  for  the  target  population.  For  example,  the  Family 
Health  area  has  reserved  50%  of  its  practice  time  to  serve  the  target 
population.  Outside  the  Centre,  community-based  (or  neighbourhood- 
based)  services  were  introduced  to  help  meet  the  needs  of  the  target 
population. 

4.  Delivery  of  Relevant/Appropriate  Services  to  the  Community 

To  a large  extent,  services  were  based  on  identified  community  need. 
This  was  accomplished  through  extensive  community  consultations  as 
well  as  review  of  epidemiological  evidence  (regional  health  status 
reports). 

5.  Delivery  of  Quality  Services  to  the  Community 

The  quest  for  measurement  of  quality  services  focused  on  the 
development  of  client  satisfaction  instruments  appropriate  for  use  with 
the  Northeast  Edmonton  population.  Clients  participating  in  focus  groups 
reported  a high  degree  of  satisfaction  with  services. 

6.  Delivery  of  Integrated  Services  to  the  Community 

One  of  the  fundamental  elements  of  the  NECHC  evaluation  and 
demonstration  projects,  as  well  as  operations  of  the  NECHC  overall,  was 
a focus  on  “integration”  (i.e.,  emphasis  on  the  vision,  goals,  and  primary 
health  care  model,  all  of  which  emphasize  integration).  Several 
processes  were  instituted  to  facilitate  integration  including,  for  example, 
a common  client  record,  centre-wide  initiatives,  comprehensive 
orientation,  and  staff  leadership  meetings. 

7.  Delivery  of  Cost-Effective  Services  to  the  Community 
The  timelines  of  the  current  project  prohibited  a formal  cost- 
effectiveness  analysis.  However,  the  internal  evaluation  team 
participated  in  discussions  that  led  to  the  development  of  a series  of 
potential  cost-effectiveness  analysis  questions  to  guide  the  development 
of  a future  economic  study. 

8.  Partnerships  are  Evident  and  Working 

Formal  agreements  were  established  with  five  external  agencies  and 
organizations.  Four  of  these  agreements  involved  the  creation  of  service 
provider  positions  at  the  NECHC  initially  set  to  end  May  31,  2000.  Most 
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of  these  are  now  ongoing  or  undergoing  negotiations  to  continue.  The 
fifth  involved  the  creation  of  an  Asthma  Centre  and  other  initiatives  for  a 
longer-term  period. 

9.  Framework  to  Measure  Potential  Impact  on  Health  Status 
Improved  health  status  and  quality  of  life  for  the  NECHC  client 
population  is  the  overall  project  goal  identified  in  the  3-5  year  evaluation 
framework.  Five  indicators  and  means  of  verification  are  provided  for 
each  health  status  outcome  (selected  outcomes  include  low  birth  weight 
babies,  heart  disease/stroke/hypertension,  diabetes,  teenage  smoking, 
and  childhood  injuries). 

Learning 

Central  Role  of  Evaluation 

■ The  project  demonstrated  that  formative  evaluation  brings  value  to 
an  organization.  Evaluation  was  perceived  as  integral  to  the 
implementation  of  a new  project. 

■ Evaluation  findings  have  guided  decision-making;  the  team  has 
illustrated  rapid  turnaround  from  findings  to  action. 

■ The  NECHC  has  developed  an  “evaluation  culture.”  Evaluation  was 
not  an  add-on  but  had  been  introduced  from  the  beginning. 

■ Importance  of  Evaluation  Coordinator  role  - liaison  between  front 
line  staff  and  management. 

■ Importance  of  dedicated  project  resources  for  evaluation  beyond 
coordinator  salary. 

Continued  Focus  on  Vision/Mission/Model 

■ Early  establishment  of  the  vision/mission  for  the  Centre  guided 
decision  making  and  maintained  focus  on  desired  outcomes. 

■ Continuing  evaluation  based  on  primary  health  care  model. 

■ Modifications  to  the  model  as  appropriate  (e.g.,  points  of  entry  are 
not  singular  through  family  health  as  originally  envisioned  but  rather 
there  are  many  points  of  entry  for  clients  to  access  services). 

ParticiDatorv  Leadership  Crucial 

■ Importance  of  effective  NECHC  leadership. 

■ Involvement  of  health  leaders  and  all  staff  in  discussion  of  evaluation 
plans  and  progress. 

■ Commitment  of  RHA  board  and  senior  administration  to  primary 
health  care  at  the  NECHC. 

Meetina  the  Needs  of  the  Communitv 

■ Services  based  on  extensive  community  consultations  as  well  as 
health  status  data.  Combined  community  perspectives  as  well  as 
epidemiological  evidence  to  support  decision-making. 

■ Ongoing  community  representation  maintained  via  the  Community 
Consultation  Committee. 

■ Community  involvement  central  to  primary  health  care  model. 

Assessina  Client  and  Provider  Satisfaction 

■ Implemented  survey  approach  to  measuring  client  satisfaction 
(based  on  literature  review).  This  was  a context-specific  and 
resource  intensive  exercise. 

■ Focus  on  customer  satisfaction  (staff/provider  training). 

■ Assessing  provider  satisfaction  through  focus  groups  and  interviews. 
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Bevond  Coordination:  Towards  Inteqration 

■ The  concept  of  integration  goes  beyond  coordination  to  include 
being  connected  at  many  different  levels. 

■ Making  integration  a reality  at  the  NECHC,  through  interdisciplinary 

collaboration,  has  been  a central  focus  for  the  Centre.  1 

■ Measurement/assessing  level  of  integration  requires  multiple 
methodologies,  including  qualitative  data. 

Choosinq  Partners  Carefullv 

■ Strategic  decision  making  regarding  entering  partnerships. 

■ Learning  how  to  work  with  the  community  and  in  the  community. 

Time  Intensive  Experiences 

■ Short  timelines  for  HTF  versus  longitudinal  nature  of  health 
outcomes. 

■ Available  resources  are  limited  and  need  to  balance  health  service 
delivery  with  evaluation  and  competing  demands. 

■ Importance  of  dedicated  individual  to  champion  evaluation  activities. 

Struoole  with  Cost-Effectiveness 

■ Challenge  to  assess  cost-effectiveness  of  interventions  while 
interventions  are  continually  changing  within  short  HTF  timelines. 

■ Difficulties  describing  resources  (costs)  and  outcomes  associated 
with  process  and  organization  of  primary  health  care  interventions; 
economic  evaluations  have  typically  focus  on  products. 

■ Difficulty  assessing  cost-effectiveness  of  interventions  when 
outcomes  are  assessed  at  the  Centre  and  costs  are  assessed  by  the 
regional  health  authority. 

Health  Status  is  the  Bottom  Line 

■ How  the  Centre  can  improve  the  health  status  of  its  target  population 
guides  decision-making. 

■ The  health  status  of  the  Northeast  Edmonton  population  (as 
determined  by  comprehensive  regional  health  status  data  reports) 
was  used  to  determine  initial  service  areas.  The  data  reports  were 
also  used  to  select  indicators  for  3-5  year  evaluation  framework. 

Contribution  to  National  Dimensions 

Access 

■ The  provision  of  neighbourhood-based  services  (initially  referred  to  as 
“outreach”)  as  well  as  those  offered  at  the  Centre  reduces  barriers  to 
access  such  as  transportation  (bringing  services  to  the  target  population). 

■ The  Community  Consultation  Committee  continues  to  address  access 
issues  with  Centre  staff. 

■ Transportation  vouchers  have  been  made  available  for  some  clients. 

Quality 

■ Developing  an  appropriate  client  satisfaction  instrument  was  a resource 
intensive  experience  and  is  context  specific.  Initial  findings  suggest  clients 
and  providers  are  satisfied  with  the  services  at  the  NECHC. 

■ More  time  is  required  to  evaluate  outcome  measures  in  a new  community 
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health  centre;  benchmarks  are  difficult  to  find. 

■ The  project  contributed  to  the  recognition  of  evidence  that  may  not  be 
quantitative  (e.g.,  quality  improvement  through  case  study). 

Integration 

■ Leadership  exemplified  and  facilitated  integration.  Board  commitment  was 
critical  for  establishing  operations  of  the  NECHC  within  requirements  of 
overall  health  system  and  regional  structure.  Site  Director  continues  to 
exemplify  leadership. 

■ Innovative  study  designs/mechanisms  are  required,  systematic  ways  to 
address  integration.  Qualitative  methods  are  appropriate. 

■ There  has  been  an  evolutionary  process  of  developing  relationships 
across  service  areas  and  among  individual  staff. 

■ There  have  been  challenges  of  integrating  existing  non-integrated 
information  systems. 

■ The  project  demonstrated  integration  with  other  parts  of  the  health  system 
and  key  stakeholders,  intentional  partnership  agreements,  and  how  to 
establish  linkages. 

■ Integration  must  be  considered  within  broader  social/political  context, 
including  among  providers  and  within  a large  regional  system. 

■ A common  (paper-based)  client  record  was  introduced  in  all  service  areas 
except  Community  Health. 

■ Centre-wide  initiatives  were  introduced  to  build  integration  among  staff 
and  to  improve  integrated  delivery  of  services. 

■ Staff  leadership  meetings  provide  further  opportunities  to  build  integration 
among  staff. 

Health  Status 

■ Determining  what  health  outcomes  to  measure  and  selecting  indicators 
was  key.  There  was  a need  to  balance  expectations  within  timeframes 
(pressure  to  illustrate  change). 

■ The  Regional  Health  Status  Report  was  seen  as  an  excellent  resource 
used  extensively  for  initial  planning  (determining  target  populations)  and 
ongoing  program  development  (introducing  new  services  based  on  health 
status  needs). 

■ The  community  consultation  process  matches  services  being  delivered 
with  need;  combining  epidemiological  evidence  of  the  Health  Status 
Report  with  the  community  perspective. 

Cost- 

Effectiveness 

■ Available  resources  and  short  timelines  prohibited  the  completion  of  a 
former  economic  evaluation. 

■ The  internal  evaluation  team  undertook  a process  of  developing  a series 
of  potential  cost-effectiveness  analysis  questions  in  consultation  with  the 
independent  evaluation  team  (guided  by  the  health  economist). 

Kev  learninqs 

■ Cost-effectiveness  is  perceived  as  difficult  to  undertake. 

■ Assessing  an  intervention  while  it  is  undergoing  rapid  change  is  difficult. 

■ Economic  evaluations  typically  focus  on  products  rather  than  on  process 
or  organizational  innovations  such  as  the  primary  health  care  model  being 
introduced  at  the  NECHC. 

■ Development  of  appropriate  questions  in  a cost-effectiveness  analysis  is 
an  important  first  step.  The  questions  are  anticipated  to  drive  the  research 
agenda. 

■ Resources  are  required  to  collect  information  to  adequately  assess  cost- 
effectiveness.  Collecting  information  on  costs  and  outcomes  is  expensive. 

■ Collection  of  cost  data  leads  to  concern  about  inappropriate  decisions  if 
health  consequences  (outcomes)  are  not  taken  into  account. 

Transferability 

■ Careful  documentation  of  evaluation  and  implementation  is  critical  for 
transferability  and  generalizability.  The  Story  document  (implementation 
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evaluation)  on  the  development  of  the  NECHC  facilitates  sharing  of 
information. 

■ Having  evaluation  in  place  from  the  beginning  and  the  use  of  formative 
evaluation  findings  to  inform  action  were  key  elements  of  the  NECHC. 

■ Essential  elements  include  a common  vision,  support  for  the  service 
delivery  model,  participatory  leadership,  and  formative  evaluation 
activities.  Co-location  of  public  health  and  relevant  primary  health  care 
services  is  also  an  important  pre-requisite  for  health  professional 
collaboration  and  integrated  service  delivery. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

Comparison  of  CHCs  with  Uraent/Emerqencv  Care  Centres 

■ The  establishment  of  Emergency  Services  within  a primary  health  care 
setting  is  a unique  aspect  of  the  Northeast  Community  Health  Centre. 
There  is  potential  to  compare  types  of  clients,  presenting  needs,  and 
outcomes  with  those  at  an  urgent  care  centre,  e.g.,  8th  and  8th,  versus 
emergency. 

Ooerationalizina  Primarv  Care 

■ The  evolution  of  the  primary  health  care  service  delivery  model  was  an 
essential  component  of  the  NECHC  projects,  that  is,  defining  and 
operationalizing  primary  health  care  within  the  Centre. 

Nurse  Practitioner 

■ One  of  the  key  ways  the  NECHC  team  envisioned  increasing  capacity  to 
deliver  primary  health  care  services  was  through  the  involvement  of  a 
nurse  practitioner  as  part  of  the  health  professional  team.  At  the  time  of 
the  final  independent  evaluation  report,  the  Centre  was  awaiting 
ministerial  approval  (which  would 

declare  Northeast  Edmonton  an  under-serviced  area)  to  allow  a nurse 
practitioner  to  practice  to  his/her  full  scope  of  practice. 

Physician  Reimbursement 

Alternate  Payment  Plans  are  in  place  for  several  service  areas  within  the 
Centre;  additional  proposals  are  pending  approval.  Recognizing  how 
physicians  are  paid  is  important  in  attempting  to  align  reimbursement  and 
incentives  with  the  primary  health  care  service  delivery  model.  Conventional 
fee-for-service  models  do  not  recognize  the  non-billable  time  a physician  may 
spend  with  clients  in  a primary  health  care  setting  (e.g.,  telephone 
consultations,  and  holistic  care). 

Implications  for 
Research  and 
Evaluation 

Information  svstems/databases 

■ Sufficient  information  systems  must  be  in  place  to  facilitate  the  ongoing 
collection  of  data  for  research  and  evaluation  purposes. 

■ Sufficient  resources  must  be  available  to  support  data  analyst  and 
research  assistant  positions. 

Central  Role  of  Evaluation 

■ Evaluation  was  introduced  at  the  beginning  of  operations  of  the  Centre. 
An  evaluation  “culture”  emerged  over  the  course  of  the  project. 

Importance  of  Evaluation  Coordinator 

■ An  evaluation  coordinator  fulfils  an  important  role  in  coordinating  the 
evaluation  activities  and  providing  the  overall  direction  and  support  for  a 
project,  including  maintaining  staff  enthusiasm  and  involvement. 
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Project  Name 

Evaluation  of  the  Alexandra  Community  Health  Centre  Internal 
Evaluation 

(Evaluation  Project) 

Project  Description 
l Expected  Outcomes 

The  Alexandra  Community  Health  Centre  has  been  in  operation  for 
twenty-six  years  serving  the  residents  of  inner-city  Calgary.  The  Centre 
operates  programs  in  three  areas:  clinical  service,  psychological 
service,  and  community  development.  The  purpose  of  the  HTF  internal 
evaluation  project  was  to  examine  the  Alexandra  CHC  as  a primary 
health  care  organization  with  respect  to  four  dimensions:  Client 
Description,  Goals,  Process,  and  Impact.  In  addition,  an  independent 
evaluation  team  was  assigned  responsibility  to  evaluate  the  internal 
evaluation  project  as  well  as  review  findings  within  the  context  of  the  six 
national  dimensions  of  primary  health  care. 

Goals  of  the  evaluation  project 

1 . Conduct  an  overall  evaluation  of  the  Alexandra  CHC  as  a primary 
health  care  organization  and  where  possible  to  develop  a 
sustainable  evaluation  plan  into  the  future; 

2.  Contribute  to  a comprehensive  understanding  of  the  role  of  primary 
health  care  in  the  Calgary  Regional  Health  Authority,  the  Province 
of  Alberta,  and  within  the  Canadian  health  care  system;  and 

3.  Contribute  to  a sustainable,  appropriate,  relevant,  and  long-term 
evaluation  strategy  of  primary  health  care  delivery  at  the  Alexandra 
CHC  and  the  impacts  on  the  health  and  social  systems  in  the 
Calgary  region. 

Objectives  of  the  evaluation  project 

1 . Conduct  an  overall  evaluation  of  primary  health  care  (PHC)  service 
delivery  to  the  Alexandra  Centre; 

2.  Develop  a health  capacity  tool  for  use  at  the  Centre; 

3.  Increase  the  long-term  evaluation  activities  at  the  Alexandra  CHC; 
and 

4.  Disseminate  information  about  PHC  service  delivery  to  others. 

Methodology  of 
Project  and 
Independent 
Evaluation 

Project  Implementation  (Internal  Evaluation) 

■ The  evaluation  framework  included  the  evaluation  of  the  three 
general  areas  of  operation:  administrative  functions,  clinical  service, 
and  community  development. 

■ Within  and  across  these  three  areas  of  operation,  there  were  four 
major  aspects  of  the  evaluation:  client  description,  goal  description, 
process  analysis,  and  impact. 

■ Six  types  of  data  collection  strategies  were  employed:  interviews, 
focus  groups,  questionnaires,  testimonials,  tracking  and  recording, 
and  document  review. 

Independent  Evaluation 
Main  data  collection  strategies 

■ Review  and  analysis  of  documentation  and  instruments  (e.g.,  the 
Independent  Evaluation  Team  provided  the  Internal  Evaluation 
Team  with  an  analysis  of  the  health  determinants  instrument  in 
relation  to  relevant  literature). 

■ Mapping  of  project  goals  and  objectives. 

■ Interviews  and  focus  groups. 

■ Ongoing  communication  with  Internal  Evaluation  Team. 

■ Observation  (“Walking  the  neighbourhoods”  served  by  the  Centre). 

■ The  Independent  Evaluation  Team  provided  consultation  and 
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support  to  the  Internal  Evaluation  Team  (e.g.,  review  and  advise  on 
methodology  issues  and  evaluation  protocols). 

Review  of 

Implementation 

Process 

■ Original  proposal  for  evaluation  of  the  Alexandra  CHC  submitted 
(Spring  1998)  by  Executive  Director  at  that  time. 

■ Independent  Evaluation  Team  contracted  (October  1998).  Meetings 
with  internal  team. 

■ New  Internal  Evaluation  Team  contracted  (March  1999). 

■ Ongoing  revisions  and  modifications  to  evaluation  design  as  a 
result  of  ongoing  staff  turnover. 

Facilitators/ 
Challenges  to 
Implementation 

■ The  CHC  had  been  in  place  for  twenty-six  years  prior  to  the 
implementation  of  the  evaluation  project. 

■ Several  key  staff  members  left  the  Alexandra  CHC  since  the 
beginning  of  the  evaluation,  including  the  original  Executive 
Director  and  subseguent  replacement,  community  development 
officer,  and  nurse  practitioner.  As  well,  the  internal  evaluators 
changed.  As  a result,  levels  of  support  varied  over  the  course  of  the 
project. 

■ Initial  challenge  regarding  reguirement  of  both  internal  evaluation 
and  independent  evaluation  components. 

■ The  independent  evaluators  identified  a number  of  issues  and 
concerns  in  the  interim  report  including  the  time  frame  of  the 
evaluation  project;  continued  viability  of  the  Alexandra  CHC;  special 
needs  of  the  population  being  served;  applicability  of  methods  and 
instruments;  availability  of  pre-existing  data;  continuity  within  the 
project;  and  delays  in  obtaining  ethics  approval.  The  independent 
evaluators  observed  that  the  CHC  tried  to  address  each  of  the 
concerns  over  the  course  of  the  project.  Primarily,  key  action 
strategies  included  redefining  the  target  population  and  deciding  to 
implement  satellite  services. 

Results 

Results  bv  Objectives 

1.  Conduct  an  overall  evaluation  of  primary  health  care  service 
delivery  to  the  Alexandra  Centre 

The  first  objective  was  fully  achieved.  A comprehensive  evaluation  was 
conducted  on  the  Alexandra  Community  Health  Centre. 

2.  Develop  a health  capacity  tool  for  use  at  the  Centre 

The  second  objective  was  modified  during  the  course  of  the  project. 
The  tool,  which  was  developed  as  part  of  a master’s  thesis  project, 
became  a Health  Determinants  Instrument  (HDI).  Initial  results  indicate 
that  the  instrument  may  prove  useful  to  the  Centre  and  to  other 
community  health  centres. 

3.  Increase  the  long-term  evaluation  activities  at  the  Alexandra 
CHC 

The  third  objective  was  met.  The  evaluation  provided  a number  of  tools 
and  revised  processes  that  will  enable  the  Centre  staff  to  continue 
monitoring  and  evaluating  both  clinical  and  community  development 
programs  in  the  future. 

4.  Disseminate  information  about  PHC  service  delivery  to  others 

The  fourth  objective  was  partially  met. 

Use  of  Findinas 

The  evaluation  identified  to  the  staff  and  the  board  that  the  focus  of  the 
CHC  has  changed  over  the  years  and  is  no  longer  relevant  to  the 
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original  target  population.  When  the  Centre  was  first  established  it  was 
to  assist  the  low-income,  ethnic-based  population  of  three  inner-city 
neighbourhoods.  At  least  two  of  the  neighbourhoods  have  since 
undergone  significant  socio-economic  changes  and  the  current 
population  of  these  neighbourhoods  is  less  likely  to  reguire  the  services 
of  the  CHC.  The  Centre  is  no  longer  well  integrated  with  the 
community.  Using  the  results  the  Alexandra  CHC  board  has  redefined 
their  target  population  and  refocused  their  service  delivery  (moving 
towards  satellite  clinics  to  better  meet  client  needs  and  holding  clinics  in 
the  evenings).  These  have  yet  to  be  implemented  and  will  reguire 
evaluation  to  ensure  the  strategies  are  meeting  identified  needs. 

Learning 

■ The  evaluation  of  the  Alexandra  CHC  has  helped  the  Centre 
refocus  its  priorities  and  services  which  should  help  maintain  the 
sustainability  of  the  Alexandra  CHC. 

■ The  requirement  for  all  funded  projects  to  be  externally  evaluated 
created  interesting  dynamics  when  the  funded  project  was  in  fact 

an  evaluation.  Initially  the  requirement  for  external  evaluators  1 

created  tension  between  the  internal  evaluators  and  the 
independent  evaluators.  However,  with  the  change  of  the  internal 
evaluators  in  early  1999  the  two  evaluation  teams  were  able  to 
develop  a good  working  relationship. 

■ The  mapping  process  (Repositioning  Project)  that  the  Alexandra 
CHC  participated  in  has  helped  them  to  identify  gaps  in  service. 

Contribution  to  National  Dimensions 

Access 

■ The  number  of  patients  seen  by  physicians  and  chiropractors  declined  in 

the  past  two  years  at  the  Alexandra  Community  Health  Centre.  The  i 

number  of  physicians  and  chiropractors  providing  services  also 
fluctuated  during  that  time  to  respond  to  changing  need. 

■ As  of  March  2000,  the  board  has  agreed  to  begin  taking  services  to 
clients  through  the  use  of  satellite  clinics  to  allow  patients  in  certain 
areas  with  transportation  difficulties  due  to  age,  infirmity,  or  location  the 
ability  to  access  services  through  the  CHC. 

■ The  board  has  also  agreed  to  refocus  the  target  population. 

Quality 

■ From  the  perspective  of  existing  clients,  it  would  appear  that  the 
Alexandra  CHC  maintains  quality  services.  Client  interviews  indicated 
that  clients  see  care  as  different  from  that  provided  in  other  service 
venues.  In  particular,  clients  viewed  staff  as  more  caring  and  more 
personable  than  elsewhere.  Clients  perceived  that  physicians  spent 
more  time  with  them  and  were  more  thorough  in  their  care.  A large 
number  of  clients  reported  that  they  did  not  seek  services  outside  the 
Alexandra  CHC  (suggesting  that  they  were  satisfied  with  services 
available  and  received  at  the  Centre). 

■ The  lack  of  continuity  of  resources  (staff  turnover,  changing  priorities) 
has  had  a major  influence  on  community  development  activities. 

■ The  internal  evaluation  report  made  recommendations  for  enhancing 
charting  and  file  maintenance  which  will  help  ensure  guality. 

■ Physicians  are  in  a process  of  developing  a means  to  self-evaluate  and 
measure  performance. 

■ The  community  development  section  of  the  CHC  is  in  the  process  of 
establishing  outcome  measures  for  community  development  activities. 

Integration 

■ The  four  major  partners  of  the  Alexandra  CHC  reported  that  partnerships 
were  formed  with  the  CHC  because  of  its  community  outreach  initiatives 
and  potential  for  optimizing  both  individual  and  community  health.  Based  | 
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on  the  results  of  the  Internal  Evaluation,  the  independent  evaluators 
report  it  would  appear  that  the  model  allows  for  integration  and 
coordination,  both  with  other  parts  of  the  health  system  and  with  other 
stakeholders.  Through  the  use  of  a multidisciplinary  approach  there 
appears  to  be  the  ability  to  integrate  and  coordinate  services  within  the 
Alexandra  CHC  and  beyond.  The  model  of  service  is  appropriate  for 
building  linkages  with  the  community.  The  Centre  takes  a broad  view  of 
health  and  places  particular  emphasis  on  the  physical,  as  well  as  social 
and  psychological  well  being  of  clients. 

Health  Status 

■ There  is  the  potential  to  systematically  assess  health  determinants  of 
patients  on  a longitudinal  basis  (Health  Determinants  Instrument). 

Cost- 

Effectiveness 

■ The  evaluation  tried  to  address  cost-effectiveness  through  a study  on 
shadow  billing.  The  Internal  Evaluators  noted  that  this  section  of  the 
evaluation  was  problematic  due  to  both  over-  and  under-reporting  of 
services  provided  by  physicians. 

■ The  Independent  Evaluators  reported  that  although  the  evaluation  did 
not  include  direct  measures  of  cost-effectiveness,  the  following 
indicators  of  cost-effectiveness  are  provided  in  the  model  of  care: 

■ The  Alexandra  CHC  is  multi-disciplinary  which  should  allow  for 
comprehensive  care  by  the  most  appropriate  provider; 

■ The  physicians  are  contract  as  opposed  to  fee-for-service  which 
generally  indicates  a willingness  to  partner  with  other  health  care 
providers  and  provide  services  by  telephone  as  appropriate;  and 

■ By  managing  patients  through  the  CHC  model,  the  physicians  are 
able  to  manage  more  difficult  cases  with  less  referral  to  specialists  or 
hospitals  - particularly  for  psychiatric  cases. 

Transferability 

■ The  Health  Determinants  Instrument  (HDI)  has  the  potential  to  provide 
useful  information  about  clients  at  the  Alexandra  CHC  on  a longitudinal 
basis.  It  also  has  the  potential  to  be  used  in  other  primary  health  care 
settings  and  provide  comparative  data. 

■ Information  regarding  the  functioning  of  the  administration  and  board 
could  be  generalized  to  other  Community  Health  Centres.  There  has 
been  a shift  at  the  Alexandra  CHC  to  recruiting  board  members  from 
outside  the  community.  Some  clients  have  indicated  this  is  resulting  in  a 
feeling  of  dislocation  from  the  needs  of  the  community.  Furthermore, 
staff  turnover  has  been  a significant  issue.  Interviews  with  staff  have 
indicated  that  some  of  the  reasons  for  dissatisfaction  have  stemmed 
from  an  unwillingness  to  let  staff  play  a role  in  important  decisions,  role 
confusion,  and  the  tendency  for  some  staff  to  take  on  too  many  roles. 
This  knowledge  can  be  transferred  to  other  primary  health  care  settings. 

■ The  codes  developed  for  non-billable  items  have  potential  for 
transferability.  During  the  shadow  billing  process  a number  of  codes 
were  developed  to  account  for  services  not  considered  billable  in  a fee- 
for-service  environment.  These  codes  provide  a record  of  services 
provided,  and  the  amount  of  the  physician’s  time  involved  in  these 
activities.  The  physicians  have  made  further  revisions  to  these  codes  in 
order  to  make  them  more  usable.  The  codes  could  be  standardized  and 
used  by  other  Community  Health  Centres  to  track  types  of  service.  The 
codes  also  have  the  ability  to  highlight  the  degree  of  interdisciplinary 
activity  occurring. 
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Contribution  to  Advancing  Primary  Health  Care  in  Alberta 


Policy 

Implications 

Physician  Payment  Plans 

■ There  was  considerable  attention  in  the  evaluation  to  the  way  the 
physicians  are  funded  (contract  versus  fee-for-service)  and  how  this 
enables  them  to  practice  in  ways  that  are  quite  unique.  By  the  end  of  the 
evaluation  project,  staff  had  incorporated  a shadow  billing  methodology 
into  their  practice  (to  express  in  financial  terms  the  amount  and  type  of 
services  provided).  Physicians  often  dealt  with  multiple  issues 
(sometimes  8 in  a single  session)  and  provided  consultation  over  the 
phone. 

Communitv  Health  Centre  Policv 

■ The  board  has  decided  to  introduce  satellite  services  to  increase  access 
to  services  and  refocus  on  its  target  population. 

OoDortunitv  for  Comoarison 

■ A common  evaluation  framework  for  CHCs  in  the  region,  or  the  province, 
would  facilitate  comparative  work  on  such  key  items  as  governance  and 
delivery  models. 

Implications  for 
Research  and 
Evaluation 

■ The  Health  Determinants  Instrument  will  continue  to  be  refined  and  will 
be  used  to  assess  health  determinants  of  patients  at  the  Centre. 

■ The  processes  developed  during  the  evaluation  will  allow  the  Centre  to 
collect  more  meaningful  data  on  a regular  basis  in  order  to  provide 
ongoing  monitoring  of  the  Centre’s  performance. 

■ The  evaluation  process  has  provided  the  staff  and  board  of  the 
Alexandra  CHC  the  opportunity  to  examine  client  and  service  data  that 
had  never  been  used  before.  This  has  provided  baseline  statistics  for 
future  review.  The  willingness  of  the  Centre  to  participate  in  and  learn 
from  the  evaluation  project  could  serve  as  a building  block  for  future 
collaborative  studies  with  other  Community  Health  Centres. 

■ It  was  necessary  to  hire  an  external  consultant  to  access  and  generate 
reports  from  the  RISE  system  (a  computer  database  that  is  widely  used 
in  Alberta  health  centres  in  place  at  the  Alexandra  CHC)  because  the 
system  was  not  amenable  to  the  type  of  report  generation  that  was 
needed  for  the  evaluation. 
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Strategy  To  Advance  Primary  Health  Care  In  Alberta  - 
System  Restructuring 


East  Central  Health  Primary  Health  Services  Initiative  Process 
Evaluation 

(Demonstration  Project) 


Project  Name 

East  Central  Health  Primary  Health  Services  Initiative  Process 
Evaluation 

Project  Description 

The  East  Central  Health  Primary  Health  Services  Initiative,  referred  to 
as  PHS,  was  a demonstration  project  intended  to  establish  integrated 
primary  health  services  in  four  rural  communities  and  surrounding 
areas  in  the  East  Central  Regional  Health  Authority  (ECH).  The  intent 
was  to  catalyze  and  support  the  evolution  of  integrated  primary  health 
services  in  the  four  rural  communities  (Consort,  Coronation,  Hardisty 
and  Castor)  and  eventually  across  the  region.  The  initiative  focused  on 
working  with  communities  in  partnership  to  build  an  integrated 
sustainable  primary  health  system  in  rural  communities. 

Expected  Outcomes 

1 . A consultative  approach  will  be  used  to  involve  community  leaders 
and  health  providers,  including  physicians,  in  the  design  and 
delivery  of  PHS. 

2.  Community  members  and  health  service  providers  including 
medical  staff  will  be  satisfied  with  their  level  of  participation  in  the 
planning  and  implementation  of  PHS  in  their  communities. 

3.  The  PHS  plan  will  reflect  a clear  relationship  of  the  proposed 
principles  of  PHS  and  will  be  consistent  with  the  goals,  objectives, 
and  strategic  directions  set  out  in  the  ECH  Business  Plan. 

4.  The  PHS  implementation  will  include  a comprehensive  range  of 
services  encompassing  preventative,  promotive,  treatment, 
supportive  and  rehabilitative  components;  improved  service 
coordination,  and  maintain  or  enhance  linkages  to  other  levels  of 
service-secondary  and  tertiary. 

5.  Ability  to  demonstrate  interdisciplinary  or  intersectoral  teams  are 
working  together. 

6.  Evidence  of  innovative  ways  for  improving  services  at  the  local 
level  which  are  responsive  to  individual/family  unmet  needs,  and 
community  health  needs. 

7.  Service  changes  will  support  communities  in  health  planning  with  a 
focus  on  the  determinants  of  health,  wellness,  and  self-reliance. 

8.  A critical  review  will  be  conducted  (assessment/evaluation)  of  the 
model  and  findings  from  the  demonstration  studies  for  application 
on  a region-wide  basis. 

9.  A process  will  be  established  for  ongoing  evaluation  of  services  and 
measurement  of  results  to  determine  how  well  he  system  is  doing 
at  the  community  and  regional  levels. 

Methodology 

There  were  two  phases  to  the  evaluation.  Phase  One  was  largely 
exploratory  in  nature.  It  captured  the  planning  and  design  phases  of  the 
initiative  at  the  regional  level  and  the  design  phase  at  the  community 
level.  Basic  concepts  were  identified  and  early  implementation  analysis 
conducted.  Phase  Two  captured  full  implementation  of  the  project.  It 
exposed  key  factors  operating  in  the  implementation  in  each 
community,  and  at  the  regional  level.  Progress  with  the  six  key  intended 
outcomes  and  challenges  identified  in  the  preliminary  phase  was 
described  and  links  to  the  national  dimensions  were  reassessed. 

For  the  purposes  of  the  process  evaluation,  the  key  areas  or  intended 
outcomes  that  were  tracked  included 

■ satisfaction  of  staff  and  communities  with  the  consultation  process; 

G extent  to  which  the  consultation  process  was  open,  engaging,  and 
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recognized  where  the  communities  were  at; 

■ clarity  of  the  relationship  between  the  PHS  Initiative  Plan,  PHS 
principles,  and  the  ECH  Business  Plan; 

■ the  extent  to  which  a continuum  of  services  was  contemplated  or 
offered  by  each  of  the  communities; 

■ establishment  and  use  of  interdisciplinary  approaches  and 
intersectional  teams  at  the  community  levels;  and 

■ existence  or  development  of  partnerships  for  health  in  each 
community,  and  between  the  region  and  the  communities. 

Data  were  collected  through 

■ on-site  observation  and  telephone  interviews; 

■ document  review,  including  regional  and  community  meeting 
minutes,  planning  reports,  newsletters,  communication  materials, 
and  media  coverage; 

■ attendance  and  observation  at  Steering  Committee  Meetings  and 
community  organizing  meetings;  and 

■ consultation  with  executive  staff  and  project  coordinators. 

The  principle  source  of  data  was  the  key  informant  interviews  held  with 
the  executive  staff,  all  steering  Committee  members  (including  the 
Medical  Director),  the  community  coordinators,  select  Board  members, 
a local  Community  Health  Council  member,  and  community 
stakeholders. 

Three  rounds  of  interviews  were  conducted. 

1 . The  first  round  of  interviews  was  limited  to  the  regional  level,  and 
internal  respondents  only  (September/October-1999). 

2.  The  second  round  of  interviews  included  the  regional  and 
community  levels,  and  still  included  internal  respondents  only 
(November/December-1999). 

3.  The  third  round  of  interviews  including  regional  and  community 
levels,  and  internal  and  external  respondents,  i.e.,  community 
stakeholders  in  each  of  the  communities  were  also  included. 

Review  of 

Implementation 

Process 

Four  main  project  phases  are  outlined 

Plannina  phase  (March-Auaust  1999) 

■ The  PHS  initiative  had  its  roots  in  strategic  directions  set  by  the 
ECH  Board  in  January  1999.  The  Board  directed  an  integrated  and 
community-based  systems-wide  approach  to  the  development  of 
primary  health  services.  Throughout  the  spring  of  1999,  executive 
staff  of  the  region  further  grounded  the  initiative  in  a series  of 
planning,  coordinating  and  facilitating  events  that  resulted  in  the 
establishment  of  a Steering  Committee.  The  region’s  commitment 
to  work  with  communities  rather  than  impose  this  new  system  set 
the  stage  for  a supportive  and  facilitative  style  rather  than  a 
directive  approach  to  involving  communities  in  the  natural  evolution 
of  primary  health  services.  There  was  also  a strong  commitment  to 
start  with  where  the  communities  were  at,  and  to  transfer  learning 
from  this  early  experience  throughout  the  rest  of  the  region.  Over 
the  next  year,  this  committee  steered  the  development  of  the 
initiative  and  supported  the  planning  and  organizing  activities  in 
each  of  the  demonstration  communities  (Consort,  Coronation, 
Hardisty,  and  Castor). 
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Desian.  Implementation  and  Evaluation  (Auqust-December  1999) 

■ In  August  1999,  the  Steering  Committee  approved  the  Evaluation 
Plan. 

Implementation  (November  1999  - May  2000) 

In  the  four  communities,  the  project  began  with  meetings  scheduled  by 
the  Region  with  the  staff,  physicians  and  community  stakeholders  in 
each  community  in  the  early  fall  1999.  By  Christmas,  all  four 
communities  had  follow-up  meetings  and  had  organized  around  PHS  in 
a fashion  meaningful  to  that  community.  By  early  spring,  each 
community  had  engaged  in  new  activity  as  a result  of  the  initiative. 
These  activities  ranged  from  service  inventories  to  multisectoral 
information  gathering,  to  special  activities,  to  planned  projects.  By  the 
end  of  the  project,  using  an  action  planning  approach,  each  community 
was  actively  reflecting  on  where  it  had  been  and  where  it  was  going  in 
the  near  future  with  PHS.  The  four  communities  selected  as  the 
demonstration  sites  shared  common  selection  criteria.  Each  of  them 

■ was  keen  on  improving  access  to  and  coordination  of  services, 

■ viewed  primary  health  services  as  a way  to  serve  people  better, 

■ displayed  a readiness  to  shift  their  focus  to  a more  integrated 
system, 

■ wanted  meaningful  involvement  in  health  planning  at  the  local  level, 

■ was  rural  in  nature,  and 

■ had  community  /facility  coordinators  who  were  respected  by  the 
community  and  had  past  experience  with  primary  health  services. 

Facilitators/ 
Challenges  to 
Implementation 

Facilitators 

■ The  four  communities  selected  as  the  demonstration  sites  shared 
common  selection  criteria  (e.g.,  each  of  them  was  keen  on 
improving  access  to  and  coordination  of  services,  viewed  primary 
health  services  as  a way  to  serve  people  better,  etc). 

■ Commitment  to  start  where  the  communities  were  at  (facilitated  by 
existing  community  and  facility  managers). 

■ Very  visible  role  of  the  President  and  CEO  as  champion  of  the 
initiative,  and  the  central  role  of  the  Vice-President  of  Health 
Services  as  Project/Initiative  Director  and  Steering  Committee 
Chair. 

Challenges 

■ A focus  on  access  at  the  community  level. 

■ Concerns  about  sustainability  by  staff  at  the  community  level. 

■ Public  confidence  in  the  system  was  weak. 

■ Although  the  region  expressed  continued  commitment  to  the  vision, 
planning,  structures,  and  activities  being  developed  by  the 
communities;  the  intensive  planning  focus  in  the  second  phase 
challenged  that  commitment  and  temporarily  strained  relations  at 
the  coordinator  and  community  levels. 

Additional  challenges  were  identified  in  the  project  report 

■ Conceptual  conflicts  (e.g.,  what  does  PHS  look  like?). 

■ Community  empowerment  conflict  (viewed  by  some  as  a way  to 
dump  responsibility  for  health  onto  the  community). 

■ Risk  of  volunteer  burnout. 

■ Strategic  alignment  of  the  organization  (e.g.,  formalized  planning, 
additional  human  resources). 

■ Understanding  new  core  business  versus  basing  primary  health 
services  on  acute  care  management  principles. 
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■ Awareness  of  how  to  implement  systems  strategies  (confusion 
regarding  what  constitutes  a systems  approach  to  health  service 
delivery). 

■ Ability  (capacity)  to  execute  the  PHS  strategy  within  short  project 
timeframes. 

Results 

1 . Satisfaction  with  consultation  process 

During  the  first  phase  of  the  evaluation,  respondents  reported  greater 
satisfaction  with  the  planning  phase  of  the  consultation  process. 
Confusion  and  tension  was  reported  during  the  design  phase.  During 
the  second  phase  of  the  evaluation,  respondents  were  very  satisfied 
with  the  consultation  process;  they  expressed  great  enjoyment,  “huge 
learning,”  and  great  new  relationships. 

2.  Extent  to  which  consultation  process  was  open,  enaaainq,  and 
recoanized  where  the  communitv  was  at 

Initially,  many  felt  that  the  region  had  its  own  agenda  and  wanted  the 
communities  to  adopt  the  same  agenda.  During  the  second  phase, 
respondents  indicated  affirmatively  that  the  region  had  realized  and 
listened  to  the  communities  and  recognized  that  each  community  would 
approach  the  initiative  in  its  own  way. 

3.  Link  with  ECH  business  plan 

Most  participants  felt  there  was  a strong  link  between  the  PHS  initiative 
and  the  ECH  Business  Plan  (during  both  phases  of  the  evaluation). 
Staff  commented  on  how  the  vision  had  not  wavered  no  matter  how 
one  addressed  the  initiative  at  this  stage.  Staff  expressed  an 
acceptance  that  it  was  really  up  to  the  communities  and  surrounding 
areas  how  this  all  fell  out.  Both  coordinators  and  community 
stakeholders,  however,  commented  on  the  extent  to  which  the  action 
plans  imposed  a seemingly  unnatural  approach  to  community 
processes  that  had  not  been  developed  with  the  same  rational  style  the 
planning  required.  Comments  were  also  offered  about  the  extent  to 
which  the  timing  of  this  process  ran  the  risk  of  imposing  organization  or 
structure  for  structure  sake,  rather  than  as  a means  to  some  end.  The 
timing  was  also  awkward  as  it  came  when  some  community  groups 
were  moving  into  the  ideas  generation  or  information  gathering  phase, 
and  found  the  planning  to  interfere  with  those  activities.  In  reflection, 
coordinators  saw  this  community-level  planning  not  as  in  conflict  with 
the  original  principles  of  PHS,  but  as  one  of  the  paradoxes  that  need  to 
be  handled  and  lived  with  in  such  initiatives. 

4.  Continuum  of  services 

Initially  there  was  confidence  that  a continuum  of  services  would 
eventually  be  available  in  all  communities  but  it  was  too  early  to  see 
this  development.  During  the  second  phase  there  was  greater 
awareness  of  what  continuum  of  services  entailed  in  each  of  the 
communities. 

5.  MultidisciDlinarv  teams  and  intersectoral  approach 

During  phase  one  concern  was  mentioned  that  the  approach  to 
physicians  in  the  Region  did  not  encourage  a multidisciplinary 
approach.  Lack  of  awareness  and  fear  were  also  mentioned  as  factors 
contributing  to  turf  protection  among  the  staff  in  general.  During  phase 
two  there  was  less  concern  about  turf  protection  and  the  capacity  to 
develop  a more  multidisciplinary  approach. 
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6.  Partnerships 

■ All  respondents  felt  partnerships  were  a large  part  of  the  vision  and 
that  the  project  and  the  activity  associated  it  seemed  like  a 
partnership.  The  key  elements  of  partnership,  according  to 
participants,  were  working  together,  planning  together,  and  being 
inclusive  (i.e.,  involving  everyone  who  wants  to  be  involved,  and 
sharing  the  work).  The  initial  meetings  or  getting  together  was 
critical,  and  community  stakeholders  suggested  the  region  be  very 
careful  about  not  taking  an  approach  that  suggests.  “We  have  an 
idea,  here’s  what  you  must  do.” 

■ The  need  to  have  physicians  involved  was  a continuing  concern. 

Coordinators  expressed  concern  that  if  the  region  did  not  move  in  a 
more  timely  fashion  to  integrate  physicians  and  the  medical  model 
into  PHS,  the  initiative  may  drift  to  where  it  becomes  integrated  into 
the  medical  model.  (The  project  report  outlines  fourteen  (14)  major 
successes  of  the  project,  beginning  with  a vision  and  philosophy  of 
primary  health  services  for  the  East  Central  Health  Region.) 


Learning 


As  identified  in  the  independent  evaluator's  report,  the  ECH  Primary 

Health  Services  Initiative  demonstrated 

■ the  importance  of  sharing  with  the  community  the  vision  of  the 
project  (what  it  is  and  what  it  isn't), 

■ the  importance  of  exploring  and  reconciling  disconnects  between 
the  project  and  regional  Business  Plan,  and 

■ strength  through  building  relations  with  communities  and  staff  on 
health  issues. 

The  following  learning  was  identified  in  the  project  report: 

■ Identify  a core  team  to  plan,  develop  and  implement  the  initiative; 

■ Plan  to  over-communicate  in  each  phase  of  the  initiative; 

■ Be  transparent  and  open  in  all  communications; 

■ Develop  a clear  vision  of  what  you  are  doing,  why  you  are  doing  it, 
where  you  are  going,  and  how  you  will  get  there; 

■ Align  organizational  roles,  strategies,  structures  and  management 
systems  with  the  vision; 

■ Ensure  commitment,  leadership  and  participation  at  all  levels  of  the 
organization  but  particularly  at  the  senior  level; 

■ Develop  strategies  to  build  relationships  in  each  community; 

■ Plan  for  the  management  of  new  concepts:  organize 
information/awareness  sessions  for  the  core  team,  staff, 
physicians,  and  community;  and 

■ Base  selection  of  potential  sites  on  criteria  for  success.  Look  for 
strong,  local  health  services  leadership,  stable  physician/medical 
services  support,  and  a community  willing  to  participate/support  the 
initiative. 


Contribution  to  National  Dimensions 


Access 


From  the  outset,  the  Primary  Health  Services  Initiative  was  seen  as  a 
way  of  improving  access  to  and  coordination  of  services  in  the  four  rural 
communities  and  the  region.  Over  the  course  of  the  project,  the 
orientation  to  access  became  increasingly  qualified  by  a concern  that 
service  gaps  be  filled,  that  duplication  be  eliminated,  and  that  access 
issues  in  the  four  communities  be  considered  in  a way  that  included 
neighboring  towns,  villages,  and  hamlets. 

The  focus  on  health  at  the  communitv  level  resulted  in  a working 
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definition  of  access  that  considered  community  capacity  for  health. 
Capacity  included  ability  to  respond  to  crises,  ability  to  deal  with 
important  issues  (e.g.,  physician  recruitment),  a range  of  health  services 
and  supports. 

Quality 

■ Attention  to  quality  during  this  project  was  tied  to  perceptions  of  access 
and  service  gaps  in  the  four  communities,  and  satisfaction  with  the 
process  used  by  the  region  and  health  providers  to  deal  with  health 
concerns. 

■ Discussion  of  the  consultation  process  also  provided  a forum  for 
identifying  ways  to  deal  with  quality  issues. 

Integration 

■ The  nature  and  intensity  of  planning  for  the  initiative  was  very 
comprehensive  and  heavily  integrated.  Strategic  directions  were  set  by 
the  Board;  links  were  made  to  the  ECH  Business  Plan  and  Alberta 
Health  and  Wellness  goals;  success  indicators  for  the  development  of 
the  integrated  primary  health  system  were  identified  by  the  region. 

■ The  project  was  facilitated  in  the  communities  by  the  existing  community 
and  facility  managers  in  the  region.  Those  staff  became  the  leadership 
glue  of  the  Initiative  inside  the  communities  and  between  the  region  and 
the  communities. 

■ Community  and  staff  stakeholders  identified  two  integration  gaps:  (1) 
lack  of  integration  of  physicians  into  the  project,  and  (2)  lack  of 
integration  of  services  based  on  the  medical  model  into  a primary  health 
services  framework. 

■ The  need  to  integrate  community  health  planning  and  service  responses 
with  the  surrounding  areas  of  the  four  communities  was  a recurring 
theme.  This  inclusiveness  and  attempt  at  cooperation  and  collaboration 
emphasized  by  the  communities  and  supported  by  the  region  was  seen 
to  have  impacted  coordination  of  services  throughout  the  project. 

■ Functional  integration  of  the  project  into  ongoing  community  health 
activities  and  integration  between  the  communities  and  the  region 
emerged  as  developmental  issues  as  the  project  progressed. 

Health  Status 

■ Significant  attention  was  paid  to  individual  and  self-responsibility  for 
health  with  an  interest  in  preventing  health  problems  to  begin  with,  and  a 
strong  commitment  to  the  use  of  partnerships  to  maintain  community 
health. 

■ There  was  a strong  awareness  of  the  link  between  economic 
development,  community  involvement  in  health  issues  and  health 
planning,  and  health  status  in  all  four  communities.  This  awareness  was 
explicitly  attributed  by  the  communities  to  the  previous  experience  with 
Healthy  Communities,  Physician  Recruitment,  and  Seniors’  Alliances. 

Cost- 

Effectiveness 

Transferability 

The  commitment  to  integrated  planning  and  to  development  of  the  PHS 
capacity  in  the  Steering  Committee  produced  a highly  functional  leadership 
and  staff  forum  that  has  key  modeling  elements  which  are  likely  transferable 
to  a variety  of  system-wide  initiatives  in  the  health  and  social  services. 

A guidebook  is  under  development.  Key  messages  that  participants  wanted 
to  leave  with  the  region  regarding  the  PHS  initiative  are  the  following: 

■ Begin  where  communities  are. 

■ Be  careful  of  too  much  jargon.  j 

■ Be  careful  to  not  over  formalize,  that  is,  force  too  much  planning  or  too 
much  organization  at  the  community  level. 

■ Let  the  natural  process  in  the  community  occur  and  be  sensitive  to  pace. 

■ Recognize  that  the  communities  have  a kind  of  expertise  that  health 
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regions  need  at  the  table  when  planning  for  integrated  primary  health 
services. 

■ Reduce  the  focus  on  hospitals  and  disease.  The  community  may  be 
more  interested  in  health  promotion  and  prevention  than  assumed. 

■ Insist  on  open,  respectful  and  partnership-oriented  approach.  It  fosters 
positive  relations  with  each  community  and  lends  credibility  to  the 
process. 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta 

Policy 

Implications 

Phvsician  incentive 

■ Incentives  to  support  physician  involvement  would  be  helpful,  although 
these  must  be  developed  at  the  community  level  and  in  consultation  with 
physician  stakeholders.  It  was  made  clear  from  the  outset  that  the 
initiative  was  not  about  paying  physicians  differently. 

Reaional  health  authoritv  Dolicv/aDDroach  to  workina  with  communities 

■ A collaborative,  participatory  approach  was  essential.  Relations  were 
strained  early  in  the  project  when  communities  perceived  that  the  region 
had  its  own  agenda.  This  was  addressed  through  an  open 
communication  and  consultation  process,  and  through  early  identification 
of  what  the  project  was  and  what  it  was  not. 

Implications  for 
Research  and 
Evaluation 

■ Process  evaluations  are  appropriate  to  assess  developmental  initiatives 
of  this  kind. 
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Project  Name 

This  demonstration  project  focused  on  identifying  areas  of  fragmentation 

and  duplication  within  the  Calgary  Regional  Health  Authority  (CRHA).  A 
"whole  systems"  (Total  Health  Process  model)  was  used  to  examine  the 
existing  system  from  three  perspectives:  population(s),  provider(s),  and 
service(s).  Over  600  services  were  mapped  creating  a visual 
representation  of  the  continuum  of  services,  providers  and  populations  with 
the  CRHA.  A balanced  scorecard  approach  was  also  used  to  assist  in  the 
development  of  goals  and  objectives  for  the  study.  Game  theory  was  used 
as  the  basis  to  create  a game  that  facilitated  the  involvement  of 

stakeholders  in  thinking  about  the  requirements  of  a new  health  system 
that  would  organize  services  to  optimize  health  and  use  resources 
efficiently  (i.e.,  in  building  consensus  around  a reconfigured  model  of 
service  delivery).  The  new  model  was  comprised  of  four  anchors:  the 
Primary  Health  Care  Anchor  (geographic,  cultural,  special  interest, 
socially/clinically  complex);  the  Systems  Anchor  (support  after  hour  care, 
specialty  episodic  care  and  administrative/infrastructure  support);  and 
Preferred  Provider  Arrangements  (single  specialty,  multi-specialty  including 
facilities,  equipment  and  supplies). 

Expected 

Outcomes 

■ To  increase  the  capacity  of  populations  to  manage  their  health. 

■ To  align  populations  with  clear  provider  accountability,  responsibility 
and  control. 

■ To  provide  the  analysis  needed  to  reposition  services  and  practices  (in 
the  CRHA)  towards  PHC. 

■ To  provide  a framework  on  which  an  integrated  PHC  system  can  be 
structured. 

■ To  address  issues  relating  to  implementation  of  a PHC  system. 

■ To  assist  in  the  development  of  policies  required  to  sustain  a PHC 
system. 

Project  Deliverables  included 

■ development  of  a Total  Health  Process; 

■ best  strategic  option,  selection  criteria,  and  a balanced  scorecard  for 
the  system  design; 

■ ideal  system  framework  for  the  provision  of  primary  health  care; 

■ policy  creation  in  the  management  of  populations  and  individuals  in 
those  populations;  and 

■ designs  outlining  the  provision  of  health  care  for  selected  multi- 
population and  single  population  groups  (application  projects). 

Methodology  of 

Independent 

Evaluation 

The  evaluation  methods  included 

■ an  examination  of  the  use  of  the  population  health  data  to  determine 
the  rigor  of  its  application  to  the  model  test; 

■ literature  review  concerning  placement  of  the  model  within  the 
conceptual  and  practice  literature  on  PHC  and  integrated  health 
services; 

■ review  of  local  project  and  other  CRHA  documents  concerning  local 
interest  and  attitudes  toward  the  project  and  its  alignment  with  or 
supplanting  of  other  approaches;  and 

- Discussions  and  interviews  with  direct  participants,  members  of 
affected  groups,  and  representatives  of  formal  organizations.  Semi- 
structured  interviews  were  with  selected  stakeholders  (N=23). 

Repositioning  of  Services 

(Demonstration  Project) 
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Review  of 

Implementation 

Process 

A whole  systems  approach  (Total  Health  Process)  was  used  to  create  a 

detailed  definition  and  framework  for  implementation.  The  following 

components  were  included. 

■ Mapping  of  health  care  services  in  CRH  A using  the  Total  Health 
Process  (THP)  model  and  stakeholder  consultation.  The  THP  model 
evaluated  existing  system  services  from  three  perspectives 
(populations,  providers  and  services)  to  develop  matrices  of 
populations  by  service  by  health  process;  service  by  population  by 
health  process;  population  by  providers  by  health  process. 

■ A balanced  scorecard  approach  was  used  to  assist  in  developing  goals 
and  objectives. 

■ Game  Theory  was  used  as  the  approach  to  involve  stakeholders 
because  it  allowed  a visual  image  of  a new  system  including  how 
component  parts  fit  together;  a decision-making  strategy  (involving 
intent,  methods  of  play,  rules  governing  action,  participants,  payoffs, 
abilities/skills  required  for  action,  interaction  patterns,  physical 
setting/environment,  required  equipment);  a way  to  achieve  consensus 
on  health  system  modeling. 

■ Strategic  options  for  the  design  of  a primary  health  care  delivery 
system  were  created  using  a variety  of  inputs.  A new  matrix  was 
developed  using  the  selected  strategic  option.  Modeling  of  the 
population  served  within  the  region  to  assist  in  the  development  of  a 
new  set  of  classifications  or  subdivisions. 

■ The  new  model  was  to  be  pilot  tested  on  the  following  selected 
populations:  multi-population  groups  (Crowfoot  clinic  and  Alexandra 
CHC);  single-population  groups  (episodic  care,  chronic  care,  baby 
delivery,  eyes,  diabetes,  stroke,  ALS). 

Facilitators/ 
Challenges  to 
Implementation 

The  model  developed  as  part  of  the  project  holds  the  potential  to  lead  to  a 
more  integrated  health  care  system  within  the  CRHA,  however,  potential 
challenges  and  barriers  to  project  success  were  identified  by  some 
respondents,  including 

■ need  for  buy-in  from  all  stakeholders,  including  the  community; 

■ resistance  to  change  from  professionals; 

■ successful  application  of  the  theoretical  construct  provided  by  the 
model; 

■ need  to  develop  a critical  mass  in  order  to  demonstrate  success;  and 

■ involvement  of  CRHA  leadership. 

Results 

■ The  project  team  spent  three  months  developing  an  understanding  of 
how  health  care  is  currently  delivered,  which  included  developing  an 
inventory  of  services. 

■ Once  identified,  these  services  were  mapped  within  the  context  of  the 
THP  model  using  three  matrices  or  pictures  of  how  services  are 
delivered.  The  first  matrix  mapped  how  groups  of  patients  and 
individuals  such  as  healthy,  episodic,  chronic  and  complex  access  and 
utilized  600  services.  The  second  matrix  mapped  what  each  existing 
service  (e.g.,  emergency,  Lung  Association,  Clinical  Nutrition,  Home 
Care,  AIDS  Calgary)  did  for  patients.  The  third  matrix  mapped  role  of 
individual  providers  in  delivery  of  health  services.  The  mapping 
exercise  provided  a useful  means  of  providing  stakeholders  with  a 
visual  representation  of  the  continuum  of  services,  providers  and 
populations  within  the  CRHA. 

■ The  “game”  was  implemented  and  played  by  200  stakeholders  for  the 
purpose  of  facilitating  thinking  about  the  requirements  for  a future 
health  system  that  both  optimizes  health  and  uses  resources 
effectively.  The  game  provided  a useful  means  of  actively  engaging 
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stakeholders  in  thinking  about  the  way  in  which  services  were  currently 
delivered  and  the  way  in  which  services  might  be  delivered  to  better 
address  the  needs  of  the  populations  served. 

■ A model  of  a reorganized  CRHA  health  care  system  was  derived  from 
the  results  of  the  game.  Four  types  of  anchors  comprised  the  new 
model  of  health  services  delivery:  the  Primary  Health  Care  Anchor 
(geographic,  cultural,  special  interest,  socially/clinically  complex);  the 
Systems  Anchor  (support  after  hour  care,  specialty  episodic  care  and 
administrative/infrastructure  support);  and  the  Preferred  Provider 
Arrangements  (single  specialty,  multi-specialty  including  facilities, 
equipment  and  supplies). 

■ Having  developed  a model,  the  local  team  then  attempted  to  test  the 
model  through  a number  of  applications.  The  team  encountered 
stakeholder  considerations  that  prevented  the  successful  establishment 
of  the  pilots  in  most  cases.  Some  analysis  of  existing  and  new  service 
delivery  model  conducted  at  some  site  with  some  groups  (ALS,  Gl 
services,  3 primary  care  sites). 

■ The  scorecard  was  developed  but  not  tested.  The  CRHA  has 
developed  and  is  using  its  own  scorecard  (not  related  to  the  scorecard 
developed  for  the  project). 

■ While  some  analysis  of  the  new  model  was  conducted  with  some 
groups  at  three  primary  care  sites,  the  model  was  not  piloted  due  to  a 
variety  of  barriers  including  lack  of  communication  with  and  support  at 
senior  levels,  difficulty  obtaining  population  health  data  to  augment  a 
revised  service  delivery  system,  and  lack  of  time  to  fully  implement  the 
new  model. 

■ The  Steering  Committee  was  disband  (unknown  reason).  A 
communications  strategy  not  developed  for  the  project.  This  may  have 
hindered  successful  implementation  of  the  model  due  to  lack  of  support 
at  senior  levels. 

■ Results  of  stakeholder  interviews  indicated  uncertainty  about  project 
status,  disconnection  from  the  region  due  to  lack  of  communication  and 
organizational  leadership/sponsorship,  and  other  more  pressing  issues 
facing  the  region.  Views  also  indicated  a possible  mis-fit  of  the  model 
with  regional  strategy,  with  the  possible  attempt  to  reframe  CRHA  into 
an  American  HMO  context.  Respondents  felt  that  the  project  had  made 
little  or  no  contribution  to  the  six  dimensions. 

■ What  we  are  left  with  is  a prototype  of  a regional  health  system  where 
some  components  have  been  partially  tested,  some  have  proved 
difficult  to  test,  and  some  have  not  been  tested. 

Learning 

Vision  and  leadership 

■ RHAs  are  complex  organizations  involving  a diverse  range  of 
stakeholders.  Thus,  implementation  of  change  is  a complicated 
process  requiring  clear  consensus  on  the  vision  of  the  organization  and 
clear  and  consistent  organizational  leadership. 

■ Strong  organizational  leader  must  champion  the  change. 

Communication 

■ Attempting  to  effect  change  on  a regional  scale  requires  a well- 
developed  and  consistent  communication  strategy.  Some,  30-40%,  of 
the  project  budget  should  have  been  devoted  to  continuous 
communication  or  project  direction  to  both  internal  and  external 
stakeholders. 

149 


Synopsis  of  Independent  Evaluation  Reports 


Barriers 

■ Once  government  has  funded  a project,  it  must  ensure  that  all  barriers 
to  successful  completion  of  the  project  are  minimized,  especially  if  the 
barriers  relate  to  government’s  own  procedures  (e.g.,  difficulties  in 
gathering  population  health  data). 

Timelines 

■ Government  funding  agencies  need  to  place  more  realistic  timelines  on 
projects  designed  to  develop,  implement  and  evaluate  new  or  existing 
innovations  in  service  delivery  or  medical  interventions  (particularly  for 
impact  and  cost-effectiveness). 

Contribution  to  National  Dimensions 

Access 

■ The  identification  of  gaps  in  service  delivery  through  the  THP  mapping 
and  matching  of  population  needs  to  resource  allocation  through  the 
population  modeling  exercise,  hold  the  potential  to  improve  access  to 
services  delivered. 

■ The  tools  of  the  model  will  be  useful  in  identifying  gaps  in  service 
delivery  and  underserved  populations,  but  the  model  will  only  function  in 
a cost-effective  manner  if  the  behavior  of  both  providers  and  patients  is 
regulated. 

Quality 

■ Data  on  quality  of  services  are  not  yet  available.  Independent  evaluators 
emphasized  that  if  the  model  developed  is  some  form  of  integrated 
delivery  system,  existing  research  does  not  clearly  support  the 
proposition  that  increased  integration  leads  to  improved  quality  of 
service. 

■ The  Total  Health  Process  (THP)  provides  a tool  for  identifying  gaps  in 
services.  The  “game”  provides  a tool  for  building  consensus  around  a 
reconfigured  model  of  service  delivery.  The  THP  mapping  and  matching 
of  population  needs  to  resource  allocation  through  population  modeling 
holds  the  potential  to  facilitate  an  improvement  in  the  quality  of  services 
delivered  (but  only  if  resources  are  reallocated  to  fill  gaps).  Patients 
would  be  managed  through  multidisciplinary  teams  (primary  health  care 
anchors)  which  are  responsible  for  managing  the  care  of  patients  across 
continuum  through  capitated  budgets.  Patients  would  register  with 
anchors.  The  mapping  exercise  and  population  modeling  allow  the 
region  to  make  decisions  about  reallocated  funding  to  populations 
demonstrating  greatest  need.  Populations  selecting  services  that  meet 
their  needs  and  taking  their  money  with  them  may  incent  providers  to 
meet  those  needs.  (Literature  does  not  support  this  necessarily, 
however.) 

Integration 

■ The  THP  model  holds  the  potential  to  facilitate  integration  and 
coordination  of  service  delivery. 

Health  Status 

■ The  model  anticipates  that  the  health  of  the  population  will  improve  by 
conceptualizing  health  through  the  lens  of  the  “total  health  process” 
which  goes  beyond  the  disease  focus  of  the  current  health  care  system. 

■ The  population-modeling  tool  holds  the  potential  to  allow  CRHA  to 
establish  a baseline  profile  for  the  service  population. 

Cost- 

Effectiveness 

■ The  combination  of  the  mapping  tool,  game,  and  population-modeling 
tool  in  theory  provide  the  means  to  match  population  needs  to  resource 
allocation.  Data  obtained  through  population  modeling  might  contribute 
to  cost-effectiveness  comparisons  over  time. 

Transferability 

■ The  project  has  developed  and  tested  tools  for  mapping  existing  health 
care  service  configurations  and  building  a consensus  around  how  best  to  1 

150 


Synopsis  of  Independent  Evaluation  Reports 


reconfigure  services;  a model  for  reconfiguring  services;  a tool  for 
modeling  population  needs;  and  a balanced  score  care  for  assessing 
goals.  Of  these  tools,  the  mapping  and  consensus  building  tools  hold  the 
most  promise  for  transferability  because  they  were  sufficiently  tested. 

The  process  of  consensus  building  through  playing  the  game  is 
transferable  because  the  game  allows  stakeholders  to  examine  existing 
service  configurations,  establish  collective  goals,  and  match  goals  to 
system  design. 

■ Arguably,  the  model  is  not  transferable  since  it  is  an  intermediate 
outcome  of  the  mapping  and  game  playing  stages.  The  model  also 
appears  to  be  based  on  assumptions  that  have  not  been  clearly 
validated  by  the  existing  body  of  knowledge  and  may  not  be  feasible 
within  the  current  regulatory  context  in  Canada.  j| 

Contribution  to  Advancing  Primary  Health  Care  in  Alberta  ;! 

Policy 

Implications 

Independent  evaluators  suggest  that  the  model  and  framework  developed  ; 

might  work  for  conceptualizing  relationships  between  stakeholders  in  either 
internal  or  quasi-private  market. 
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NATIONAL  DIMENSIONS  OF  INQUIRY 


Quality  of  Service 

1.  How  does  this  model  or  program  affect  the  quality  of  services  provided? 

■ changes  in  satisfaction  of  various  stakeholders  with  the  quality  of  service  of 
patient/families;  the  larger  community;  providers/practitioners; 
administrators/management 

■ changes  in  satisfaction  with  the  quality  of  patient/provider  relationship  (i.e., 

“caring  dimensions  of  service) 

■ changes  in  technical  appropriateness  of  intervention 

■ changes  in  availability  and  use  of  relevant  resources 

■ matching  patient  needs  to  the  most  appropriate  provider 

Access 

2.  How  does  this  model  or  program  affect  access  to  health  services? 

■ changes  in  access  to  available  services 

■ changes  in  access  to  service/preferred  provider 

■ changes  in  waiting  times 

■ changes  in  the  provision  of  culturally  and  linguistically  sensitive  services 

■ changes  in  distribution  of  access  by  gender,  geographic  location,  and/or  ethnic 
group 

■ changes  in  location  or  accommodations  where  services  are  provided  (e.g.,  co- 
locate in  a seniors  center  or  youth  center 

Integration 

3.  (a)  In  what  ways  does  this  model/program  facilitate  integration/co-ordination 
with  other  parts  of  the  health  system  and  other  health  stakeholders? 

■ changes  in  the  mix  of  services  or  nature  of  services 

■ changes  in  use  and  mix  of  existing  resources,  including  availability  and  use  of 
relevant  resources 

■ development  of  better  ways  of  gathering  data,  sharing  information  and/or 
communicating  through  the  system 

■ potential  responsiveness  to  emerging  health  issues  (relationship  to  policy, 
planning,  and  governance  infrastructures) 

■ evidence  of  mechanisms  for  integration/co-ordination  (e.g.,  central  committees, 
integrated  planning,  letters  of  agreement,  program-based  partnerships,  co- 
locating or  inter-related  services,  cross-training  of  staff,  multidisciplinary  health 
service  teams) 

■ avoidance  of  overlap  or  duplication  of  service/care 

(b)  If  services  that  are  not  normally  publicly  funded  were  provided  as  part  of 
the  model  piloted  or  program  evaluated,  how  well  integrated  was  their  delivery 
with  the  delivery  of  publicly  funded  services? 

(c)  If  services  not  insured  under  the  Canada  Health  Act  (CHA)  were  provided  as 
part  of  the  model  piloted  or  program  evaluated,  how  well  integrated  was  their 
deliver  with  the  delivery  of  services  that  are  insure  under  the  CHA?  In 
particular,  to  what  degree  does  their  delivery  respect/compliment  the 


The  Alberta  Primary  Health  Care  Project 


1 


principles  of  the  CHA?  (Services  insured  under  the  Canada  Health  Act  must 
respect  the  principles  of  accessibility,  portability,  comprehensiveness, 
universality,  and  public  administration.) 

Health  Impacts/Effects 

4.  What  kinds  of  changes  in  the  health  of  your  service/target  population  occurred 
as  a result  of  your  project,  and  on  what  basis  did  you  draw  these  conclusions? 

■ changes  to  measurable  health  status  indicators:  changes  in  morbidity,  i.e., 
decline  in  incidence  rates  per  1,000  population;  harm  reduction  or  secondary 
symptom  reduction  [given  the  time  frame  of  the  HTF-funded  projects  - maximum 
2 years  - may  need  to  use  proxies  for  health  status  indicators] 

■ population  perspective  on  changes  to  health  status  (i.e.,  client  assessment) 

■ unintended  (or  unanticipated)  results  - for  example,  project  not  reaching  a 
relevant  population  or  reaching  an  unintended  population,  or  unanticipated 
impacts  on  other  aspects  of  the  health  system 

Cost-Effectiveness 

5.  How  did  the  model  piloted  or  program  evaluated  contribute  to  a more  cost- 
effective  service  than  what  is  currently  being  provided  in  that  region/province 

■ changes/differences  in  effectiveness 

■ changes/differences  in  costs  (e.g.,  decrease/increase  in  per  client  (unit)  costs  for 
services) 

■ use  of  more  appropriate  mix  of  providers  in  more  appropriate  settings 

Transferability/Generalizability 

6.  What  lessons  did  you  learn  about  implementing  and  testing  this  model  or 
program  that  might  be  useful  to  other  jurisdictions/regions/programs/settings? 

■ What  relationships  were  found  to  exist  between  the  characteristics  of 
implementation  of  this  project  and  the  outcomes  of  this  project? 

■ What  system  pre-requisites  for  success  were  identified? 

■ What  obstacles  to  success  were  encountered? 

■ What  was  innovative  about  the  model  or  program? 

■ In  what  ways  did  integration/co-ordination  of  services  help  or  hinder  the 
achievement  of  overall  project  goals? 

■ What  elements  of  this  project  seem  particularly  transferable  to  other 
jurisdictions? 

■ How  generalizable  is  this  project,  i.e.,  would  this  project  be  applicable  in  other 
settings,  and  what  is  needed  to  make  it  applicable? 

■ Which  population  groups  benefited  the  most  from  this  model  or  program? 

■ How  did  this  model  or  program  impact  on  the  utilization  of  non-health  social 
services? 
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Guidelines  for  Final  Project  Evaluation  Report 

The  focus  of  the  final  report  for  independent  evaluators  is  two-fold:  first  at  the  project  level, 
second  at  the  national  level  of  inquiry.  The  components  of  the  report  are  addressed  below. 
Again,  you  may  wish  to  organize  your  final  report  this  way,  although  your  creativity  is  valued. 

Basic  Components  of  Final  Evaluation  Reports 

1 . Context 

2.  Description  of  project  including  intended  goals  and  outcomes 

3.  Implementation  approach 

Methodologies 

• Were  they  appropriate? 

• What  would  have  been  better?  Why? 

• Was  there  anything  innovative? 

Methods  (with  respect  to  data  collection  strategies,  analysis,  synthesis, 
verification) 

• What  were  used  (internally,  independently)? 

• Were  they  a good  choice?  Why/why  not? 

• Was  there  anything  innovative? 

4.  Evaluation  approach  (Methodologies/ Methods  as  above) 

5.  Outcomes  of  implementation 

6.  Findings  (including  assessment  of  the  extent  to  which  intended  project  outcomes  were 
achieved,  identification  of  unintended  outcomes,  identification  of  innovations,  uniqueness) 

7.  Learning  (e.g.,  How  did  the  project  impact  the  system?  What  barriers/challenges  were 
experienced?  How  were  they  addressed/overcome?) 

8.  Next  Steps?  (What  findings  do  you  think  should  be  disseminated  to  stakeholder  groups?) 

Contribution  to  National  Dimensions.  This  part  of  the  report  should  be  comprehensive.  That 
is,  review  the  list  of  questions  under  each  of  the  six  dimensions.  Address  as  many  that 
apply  to  your  project.  Also,  it  is  important  to  think  about  how  the  findings  inter-relate. 

Other  Questions  for  Reflection 

• How  does  your  project/findings  push  the  edges? 

• Did  the  project  demonstrate  a particular  life  stage  of  development/concepts/strategies? 

• How  has  the  project  encouraged/enhanced  “system-wide  thinking”  about  primary  health 
care  (i.e.,  integrated  service  delivery)? 

• What  is  the  impact  on  acute  and  social  care? 

• What  is  the  match  between  rewards  and  incentives? 

• Where  are  the  politicians? 
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• Where  is  risk  aversion  evident?  What  is  the  result? 

• What  “forces  of  change”  have  been  brought  to  bear  on  facilitating  a focus  on  primary 
health  care  or  movement  toward  it? 

• Policy  implications  (you  may  want  to  refer  to  the  Draft  Discussion  Paper).  How  do 
project  findings  inform/enhance/expand  what  is  addressed  in  the  paper?  (If  appropriate 
include  some  of  this  thinking  in  your  report). 

Process 

Draft  Final  Evaluation  Reports  should  be  submitted  to  your  project  participants  for  their  review 
and  verification.  With  incorporation  of  project  participants  feedback  (where  appropriate),  a Final 
Evaluation  Report  should  then  be  submitted  to  Howard  Research.  The  Evaluation  Management 
Team  will  then  review  it  and  accept  it  and/or  request  revisions  and  re-submission.  The  following 
important  timelines  apply: 

Submission  of  Draft  Evaluation  Final  Report  to  Project  May  2000 

Submission  of  Final  Evaluation  Report  to  Howard  Research  June  30,  2000 

If  your  project  ends  prior  to  May  30,  2000,  please  feel  free  to  submit  your  Final  Evaluation 
Report  as  soon  as  possible,  but  not  later  than  June  30,  2000. 
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